External/DevOps-Security-Agent-Skills
1
0
Fork 0
mirror of https://github.com/BagelHole/DevOps-Security-Agent-Skills.git synced 2026-08-22 12:49:53 +02:00
Code Issues Packages Projects Releases Wiki Activity

V2

Browse Source
This commit is contained in:
Toby
2026-03-24 18:02:50 -04:00
parent 2d209b9258
commit ba9e489584
111 changed files with 48382 additions and 3099 deletions
Download Patch File Download Diff File Expand all files Collapse all files
compliance/auditing/audit-logging/SKILL.md
+413 -42
View File
@@ -9,74 +9,445 @@ metadata:
# Audit Logging
Implement comprehensive audit logging for compliance.
Implement comprehensive audit logging for compliance, security monitoring, and forensic analysis across infrastructure and applications.
## When to Use
- Setting up centralized logging for compliance frameworks (SOC 2, HIPAA, PCI DSS)
- Implementing security event monitoring and alerting
- Building audit trails for regulatory requirements
- Configuring log retention and tamper-proof storage
- Integrating application logs with SIEM platforms
## Log Categories
```yaml
audit_events:
authentication:
- Login attempts
- MFA events
- Session management
- Login attempts (success and failure)
- MFA enrollment and verification events
- Session creation, renewal, and termination
- Password changes and resets
- API key and token generation
authorization:
- Access grants
- Permission changes
- Role assignments
- Access grants and denials
- Permission changes and role assignments
- Privilege escalation events
- Resource sharing modifications
- Policy evaluation results
data_access:
- Read operations
- Write operations
- Delete operations
- Read operations on sensitive data
- Write and update operations
- Delete and purge operations
- Bulk export and download events
- Data classification changes
administrative:
- Configuration changes
- User management
- System changes
- User and group management
- System startup and shutdown
- Backup and restore operations
- Network and firewall rule changes
system:
- Service health state changes
- Resource provisioning and deprovisioning
- Certificate and key rotation events
- Scheduled job execution results
- Integration and webhook events
```
## Application Logging
## Rsyslog Configuration for Centralized Logging
```bash
# /etc/rsyslog.d/50-audit.conf
# Load imfile module to read application logs
module(load="imfile")
# Forward auth logs
input(type="imfile"
File="/var/log/auth.log"
Tag="auth"
Severity="info"
Facility="auth"
)
# Forward application audit logs
input(type="imfile"
File="/var/log/app/audit.log"
Tag="app-audit"
Severity="info"
Facility="local0"
)
# Structured JSON template
template(name="json-audit" type="list") {
constant(value="{")
constant(value="\"@timestamp\":\"") property(name="timereported" dateFormat="rfc3339")
constant(value="\",\"host\":\"") property(name="hostname")
constant(value="\",\"severity\":\"") property(name="syslogseverity-text")
constant(value="\",\"facility\":\"") property(name="syslogfacility-text")
constant(value="\",\"tag\":\"") property(name="syslogtag" format="json")
constant(value="\",\"message\":\"") property(name="msg" format="json")
constant(value="\"}\n")
}
# Forward to central syslog server over TLS
action(
type="omfwd"
target="syslog.internal.example.com"
port="6514"
protocol="tcp"
StreamDriver="gtls"
StreamDriverMode="1"
StreamDriverAuthMode="x509/name"
template="json-audit"
queue.type="LinkedList"
queue.size="50000"
queue.filename="fwd_audit"
queue.saveonshutdown="on"
action.resumeRetryCount="-1"
)
```
## Journald Configuration for Persistent Logging
```ini
# /etc/systemd/journald.conf
[Journal]
Storage=persistent
Compress=yes
Seal=yes
SplitMode=uid
MaxRetentionSec=365d
MaxFileSec=30d
SystemMaxUse=10G
SystemKeepFree=2G
ForwardToSyslog=yes
```
```bash
# Query journald for audit events
journalctl _TRANSPORT=audit --since "24 hours ago" --output json-pretty
# Filter by specific audit types
journalctl _AUDIT_TYPE=1112 --since today # user login events
journalctl _AUDIT_TYPE=1100 --since today # user auth events
# Export for offline analysis
journalctl --since "7 days ago" --output export > /backup/journal-export.bin
```
## Application Logging with Structured JSON
```python
import logging
import json
import hashlib
from datetime import datetime, timezone
from functools import wraps
class AuditLogger:
def log_event(self, event_type, user, resource, action, result):
def __init__(self, service_name, logger_name="audit"):
self.service = service_name
self.logger = logging.getLogger(logger_name)
handler = logging.FileHandler("/var/log/app/audit.log")
handler.setFormatter(logging.Formatter("%(message)s"))
self.logger.addHandler(handler)
self.logger.setLevel(logging.INFO)
self._prev_hash = None
def log_event(self, event_type, user, resource, action, result,
metadata=None, source_ip=None):
log_entry = {
'timestamp': datetime.utcnow().isoformat(),
'event_type': event_type,
'user': user,
'resource': resource,
'action': action,
'result': result,
'source_ip': request.remote_addr
"timestamp": datetime.now(timezone.utc).isoformat(),
"service": self.service,
"event_type": event_type,
"user": user,
"resource": resource,
"action": action,
"result": result,
"source_ip": source_ip,
"metadata": metadata or {},
}
logger.info(json.dumps(log_entry))
# Chain hash for tamper detection
raw = json.dumps(log_entry, sort_keys=True)
log_entry["prev_hash"] = self._prev_hash
log_entry["hash"] = hashlib.sha256(
f"{self._prev_hash}:{raw}".encode()
).hexdigest()
self._prev_hash = log_entry["hash"]
self.logger.info(json.dumps(log_entry))
def log_auth(self, user, action, success, source_ip=None, mfa=False):
self.log_event(
event_type="authentication",
user=user,
resource="auth-service",
action=action,
result="success" if success else "failure",
metadata={"mfa_used": mfa},
source_ip=source_ip,
)
def log_data_access(self, user, resource, operation, record_count=0,
source_ip=None):
self.log_event(
event_type="data_access",
user=user,
resource=resource,
action=operation,
result="success",
metadata={"record_count": record_count},
source_ip=source_ip,
)
def audit_trail(audit_logger, resource_name):
"""Decorator to automatically audit function calls."""
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
user = kwargs.get("current_user", "system")
try:
result = func(*args, **kwargs)
audit_logger.log_event(
event_type="operation",
user=user,
resource=resource_name,
action=func.__name__,
result="success",
)
return result
except Exception as e:
audit_logger.log_event(
event_type="operation",
user=user,
resource=resource_name,
action=func.__name__,
result="failure",
metadata={"error": str(e)},
)
raise
return wrapper
return decorator
```
## Centralized Logging
## Fluentd / Fluent Bit Log Aggregation
```yaml
# Fluentd configuration
<source>
@type tail
path /var/log/audit/*.log
tag audit.*
</source>
# fluent-bit.conf - lightweight agent on each node
[SERVICE]
Flush 5
Daemon Off
Log_Level info
Parsers_File parsers.conf
<match audit.**>
@type elasticsearch
host elasticsearch.example.com
index_name audit-logs
</match>
[INPUT]
Name tail
Path /var/log/app/audit.log
Parser json
Tag audit.app
Refresh_Interval 5
Rotate_Wait 30
[INPUT]
Name systemd
Tag audit.system
Systemd_Filter _TRANSPORT=audit
[FILTER]
Name modify
Match audit.*
Add cluster ${CLUSTER_NAME}
Add node ${NODE_NAME}
[OUTPUT]
Name es
Match audit.*
Host elasticsearch.internal.example.com
Port 9200
Index audit-logs
Type _doc
tls On
tls.verify On
Retry_Limit 5
[OUTPUT]
Name s3
Match audit.*
region us-east-1
bucket audit-logs-archive
total_file_size 50M
upload_timeout 10m
s3_key_format /logs/%Y/%m/%d/$TAG/%H_%M_%S.gz
compression gzip
```
## Elasticsearch Index Lifecycle for Retention
```json
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_size": "50gb",
"max_age": "1d"
},
"set_priority": { "priority": 100 }
}
},
"warm": {
"min_age": "7d",
"actions": {
"shrink": { "number_of_shards": 1 },
"forcemerge": { "max_num_segments": 1 },
"set_priority": { "priority": 50 }
}
},
"cold": {
"min_age": "30d",
"actions": {
"freeze": {},
"set_priority": { "priority": 0 }
}
},
"delete": {
"min_age": "365d",
"actions": { "delete": {} }
}
}
}
}
```
## Retention Policy by Compliance Framework
```yaml
retention_requirements:
soc2:
minimum: 1 year
recommended: 3 years
notes: "Based on audit period and report requirements"
hipaa:
minimum: 6 years
notes: "From date of creation or last effective date"
pci_dss:
minimum: 1 year
immediately_available: 3 months
notes: "Req 10.7 - retain for at least one year, 3 months immediately available"
gdpr:
minimum: "As long as necessary for processing purpose"
notes: "Apply data minimization; delete when no longer needed"
fedramp:
minimum: 3 years
notes: "AU-11 control requirement"
iso27001:
minimum: "Defined by organization policy"
recommended: 3 years
notes: "A.12.4.1 - retention period must be defined"
```
## Log Integrity Verification Script
```bash
#!/usr/bin/env bash
# verify-log-integrity.sh - Verify log file checksums against stored hashes
LOG_DIR="/var/log/app"
HASH_FILE="/var/log/app/.checksums"
ALERT_WEBHOOK="${ALERT_WEBHOOK_URL}"
verify_logs() {
local failures=0
while IFS=' ' read -r stored_hash filename; do
if [ -f "$filename" ]; then
current_hash=$(sha256sum "$filename" | awk '{print $1}')
if [ "$stored_hash" != "$current_hash" ]; then
echo "TAMPER DETECTED: $filename"
failures=$((failures + 1))
curl -s -X POST "$ALERT_WEBHOOK" \
-H "Content-Type: application/json" \
-d "{\"text\":\"ALERT: Audit log tamper detected on $(hostname): $filename\"}"
fi
else
echo "MISSING: $filename"
failures=$((failures + 1))
fi
done < "$HASH_FILE"
return $failures
}
update_checksums() {
find "$LOG_DIR" -name "*.log" -type f -exec sha256sum {} \; > "$HASH_FILE"
chmod 440 "$HASH_FILE"
}
case "${1:-verify}" in
verify) verify_logs ;;
update) update_checksums ;;
*) echo "Usage: $0 {verify|update}" ;;
esac
```
## SIEM Integration Checklist
```yaml
siem_integration:
log_sources:
- [ ] Operating system auth logs (syslog, journald)
- [ ] Application audit logs (structured JSON)
- [ ] Cloud provider audit trails (CloudTrail, Activity Log, Audit Logs)
- [ ] Database query and access logs
- [ ] Network flow logs and firewall logs
- [ ] Container and orchestrator logs (Kubernetes audit)
- [ ] WAF and CDN access logs
- [ ] VPN and remote access logs
normalization:
- [ ] Common event format (CEF) or OCSF schema
- [ ] Consistent timestamp format (ISO 8601 / UTC)
- [ ] Unified user identity fields
- [ ] Standardized severity levels
alerting_rules:
- [ ] Multiple failed login attempts (brute force)
- [ ] Login from unusual location or device
- [ ] Privilege escalation events
- [ ] Sensitive data bulk export
- [ ] Administrative action outside change window
- [ ] Service account anomalous activity
- [ ] Log forwarding gap or interruption
operational:
- [ ] Log pipeline health monitoring
- [ ] Storage capacity alerting
- [ ] Retention policy enforcement verified
- [ ] Backup of log archives confirmed
- [ ] Access to log systems restricted and audited
```
## Best Practices
- Structured logging (JSON)
- Centralized collection
- Tamper-proof storage
- Retention policies
- Alerting on anomalies
- Use structured logging (JSON) with consistent field names across all services
- Ship logs to a centralized platform with write-once storage for tamper protection
- Implement hash chaining or digital signatures for log integrity verification
- Define and enforce retention policies per compliance framework requirements
- Set up real-time alerting for high-severity security events
- Separate audit logs from application debug logs to reduce noise
- Never log sensitive data (passwords, tokens, PII) in audit entries
- Monitor the logging pipeline itself to detect gaps in coverage
- Regularly test log restoration from archives to verify recoverability
- Rotate and compress logs to manage storage while meeting retention windows
compliance/auditing/aws-cloudtrail/SKILL.md
+425 -26
View File
@@ -9,56 +9,455 @@ metadata:
# AWS CloudTrail
Audit AWS account activity with CloudTrail.
Audit AWS account activity with CloudTrail for compliance, security investigation, and operational troubleshooting.
## Create Trail
## When to Use
- Enabling organization-wide audit logging across all AWS accounts
- Investigating security incidents or unauthorized API activity
- Meeting compliance requirements for SOC 2, HIPAA, PCI DSS, or FedRAMP
- Setting up automated alerting on sensitive AWS API calls
- Querying historical AWS activity for forensic analysis
## Create an Organization Trail
```bash
# Create organization trail
# Create the S3 bucket for log storage
aws s3api create-bucket \
--bucket org-cloudtrail-audit-logs \
--region us-east-1
# Apply bucket policy allowing CloudTrail to write
aws s3api put-bucket-policy \
--bucket org-cloudtrail-audit-logs \
--policy '{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck",
"Effect": "Allow",
"Principal": {"Service": "cloudtrail.amazonaws.com"},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::org-cloudtrail-audit-logs"
},
{
"Sid": "AWSCloudTrailWrite",
"Effect": "Allow",
"Principal": {"Service": "cloudtrail.amazonaws.com"},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::org-cloudtrail-audit-logs/AWSLogs/*",
"Condition": {
"StringEquals": {"s3:x-amz-acl": "bucket-owner-full-control"}
}
}
]
}'
# Block public access on the audit bucket
aws s3api put-public-access-block \
--bucket org-cloudtrail-audit-logs \
--public-access-block-configuration \
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
# Enable versioning for tamper protection
aws s3api put-bucket-versioning \
--bucket org-cloudtrail-audit-logs \
--versioning-configuration Status=Enabled
# Enable server-side encryption
aws s3api put-bucket-encryption \
--bucket org-cloudtrail-audit-logs \
--server-side-encryption-configuration '{
"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "aws:kms", "KMSMasterKeyID": "alias/cloudtrail-key"}}]
}'
# Set lifecycle policy for log retention
aws s3api put-bucket-lifecycle-configuration \
--bucket org-cloudtrail-audit-logs \
--lifecycle-configuration '{
"Rules": [
{
"ID": "TransitionToGlacier",
"Status": "Enabled",
"Filter": {"Prefix": "AWSLogs/"},
"Transitions": [
{"Days": 90, "StorageClass": "GLACIER"}
]
},
{
"ID": "ExpireOldLogs",
"Status": "Enabled",
"Filter": {"Prefix": "AWSLogs/"},
"Expiration": {"Days": 2555}
}
]
}'
# Create the organization trail
aws cloudtrail create-trail \
--name org-audit-trail \
--s3-bucket-name audit-logs-bucket \
--s3-bucket-name org-cloudtrail-audit-logs \
--is-organization-trail \
--is-multi-region-trail \
--enable-log-file-validation \
--kms-key-id arn:aws:kms:...
--kms-key-id arn:aws:kms:us-east-1:123456789012:alias/cloudtrail-key \
--cloud-watch-logs-log-group-arn arn:aws:logs:us-east-1:123456789012:log-group:CloudTrail:* \
--cloud-watch-logs-role-arn arn:aws:iam::123456789012:role/CloudTrail-CWLogs-Role
# Start logging
aws cloudtrail start-logging --name org-audit-trail
```
## Event Selectors
## Event Selectors for Management and Data Events
```bash
# Log all management and data events
# Configure advanced event selectors for granular control
aws cloudtrail put-event-selectors \
--trail-name org-audit-trail \
--event-selectors '[{
"ReadWriteType": "All",
"IncludeManagementEvents": true,
"DataResources": [{
"Type": "AWS::S3::Object",
"Values": ["arn:aws:s3:::sensitive-bucket/"]
}]
}]'
--advanced-event-selectors '[
{
"Name": "AllManagementEvents",
"FieldSelectors": [
{"Field": "eventCategory", "Equals": ["Management"]}
]
},
{
"Name": "S3DataEventsForSensitiveBuckets",
"FieldSelectors": [
{"Field": "eventCategory", "Equals": ["Data"]},
{"Field": "resources.type", "Equals": ["AWS::S3::Object"]},
{"Field": "resources.ARN", "StartsWith": [
"arn:aws:s3:::sensitive-data-bucket/",
"arn:aws:s3:::pii-bucket/",
"arn:aws:s3:::financial-data/"
]}
]
},
{
"Name": "LambdaInvocations",
"FieldSelectors": [
{"Field": "eventCategory", "Equals": ["Data"]},
{"Field": "resources.type", "Equals": ["AWS::Lambda::Function"]}
]
},
{
"Name": "DynamoDBDataEvents",
"FieldSelectors": [
{"Field": "eventCategory", "Equals": ["Data"]},
{"Field": "resources.type", "Equals": ["AWS::DynamoDB::Table"]}
]
}
]'
```
## CloudTrail Lake
## CloudWatch Alerts for Sensitive Activity
```bash
# Create metric filter for unauthorized API calls
aws logs put-metric-filter \
--log-group-name CloudTrail \
--filter-name UnauthorizedAPICalls \
--filter-pattern '{ ($.errorCode = "*UnauthorizedAccess*") || ($.errorCode = "AccessDenied*") }' \
--metric-transformations \
metricName=UnauthorizedAPICalls,metricNamespace=CloudTrailMetrics,metricValue=1
# Create alarm for unauthorized calls
aws cloudwatch put-metric-alarm \
--alarm-name UnauthorizedAPICallsAlarm \
--metric-name UnauthorizedAPICalls \
--namespace CloudTrailMetrics \
--statistic Sum \
--period 300 \
--threshold 5 \
--comparison-operator GreaterThanOrEqualToThreshold \
--evaluation-periods 1 \
--alarm-actions arn:aws:sns:us-east-1:123456789012:security-alerts
# Root account usage alarm
aws logs put-metric-filter \
--log-group-name CloudTrail \
--filter-name RootAccountUsage \
--filter-pattern '{ ($.userIdentity.type = "Root") && ($.userIdentity.invokedBy NOT EXISTS) && ($.eventType != "AwsServiceEvent") }' \
--metric-transformations \
metricName=RootAccountUsage,metricNamespace=CloudTrailMetrics,metricValue=1
aws cloudwatch put-metric-alarm \
--alarm-name RootAccountUsageAlarm \
--metric-name RootAccountUsage \
--namespace CloudTrailMetrics \
--statistic Sum \
--period 300 \
--threshold 1 \
--comparison-operator GreaterThanOrEqualToThreshold \
--evaluation-periods 1 \
--alarm-actions arn:aws:sns:us-east-1:123456789012:security-alerts
# Console login without MFA
aws logs put-metric-filter \
--log-group-name CloudTrail \
--filter-name ConsoleLoginWithoutMFA \
--filter-pattern '{ ($.eventName = "ConsoleLogin") && ($.additionalEventData.MFAUsed != "Yes") && ($.userIdentity.type = "IAMUser") }' \
--metric-transformations \
metricName=ConsoleLoginWithoutMFA,metricNamespace=CloudTrailMetrics,metricValue=1
# IAM policy changes
aws logs put-metric-filter \
--log-group-name CloudTrail \
--filter-name IAMPolicyChanges \
--filter-pattern '{ ($.eventName=CreatePolicy) || ($.eventName=DeletePolicy) || ($.eventName=AttachRolePolicy) || ($.eventName=DetachRolePolicy) || ($.eventName=AttachUserPolicy) || ($.eventName=PutUserPolicy) }' \
--metric-transformations \
metricName=IAMPolicyChanges,metricNamespace=CloudTrailMetrics,metricValue=1
# Security group changes
aws logs put-metric-filter \
--log-group-name CloudTrail \
--filter-name SecurityGroupChanges \
--filter-pattern '{ ($.eventName=AuthorizeSecurityGroupIngress) || ($.eventName=RevokeSecurityGroupIngress) || ($.eventName=CreateSecurityGroup) || ($.eventName=DeleteSecurityGroup) }' \
--metric-transformations \
metricName=SecurityGroupChanges,metricNamespace=CloudTrailMetrics,metricValue=1
```
## Athena Queries for CloudTrail Analysis
```sql
-- Query events
SELECT eventTime, userIdentity.userName, eventName, sourceIPAddress
-- Create Athena table for CloudTrail logs
CREATE EXTERNAL TABLE IF NOT EXISTS cloudtrail_logs (
eventVersion STRING,
userIdentity STRUCT<
type: STRING,
principalId: STRING,
arn: STRING,
accountId: STRING,
invokedBy: STRING,
accessKeyId: STRING,
userName: STRING,
sessionContext: STRUCT<
attributes: STRUCT<mfaAuthenticated: STRING, creationDate: STRING>,
sessionIssuer: STRUCT<type: STRING, principalId: STRING, arn: STRING, accountId: STRING, userName: STRING>
>
>,
eventTime STRING,
eventSource STRING,
eventName STRING,
awsRegion STRING,
sourceIPAddress STRING,
userAgent STRING,
errorCode STRING,
errorMessage STRING,
requestParameters STRING,
responseElements STRING,
additionalEventData STRING,
requestId STRING,
eventId STRING,
readOnly STRING,
resources ARRAY<STRUCT<arn: STRING, accountId: STRING, type: STRING>>,
eventType STRING,
recipientAccountId STRING
)
PARTITIONED BY (region STRING, year STRING, month STRING, day STRING)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
LOCATION 's3://org-cloudtrail-audit-logs/AWSLogs/123456789012/CloudTrail/';
-- Find all delete operations in the last 7 days
SELECT eventTime, userIdentity.arn, eventName, sourceIPAddress,
requestParameters
FROM cloudtrail_logs
WHERE eventTime > '2024-01-01'
AND eventName LIKE '%Delete%'
WHERE eventName LIKE '%Delete%'
AND eventTime > date_format(date_add('day', -7, current_date), '%Y-%m-%dT%H:%i:%sZ')
ORDER BY eventTime DESC
LIMIT 100
LIMIT 100;
-- Identify console logins from unusual IP addresses
SELECT eventTime, userIdentity.userName, sourceIPAddress,
additionalEventData
FROM cloudtrail_logs
WHERE eventName = 'ConsoleLogin'
AND sourceIPAddress NOT IN ('198.51.100.0/24', '203.0.113.0/24')
AND eventTime > date_format(date_add('day', -30, current_date), '%Y-%m-%dT%H:%i:%sZ')
ORDER BY eventTime DESC;
-- Access key usage patterns per principal
SELECT userIdentity.arn,
count(*) AS api_call_count,
count(DISTINCT eventName) AS unique_actions,
count(DISTINCT sourceIPAddress) AS unique_ips,
min(eventTime) AS first_seen,
max(eventTime) AS last_seen
FROM cloudtrail_logs
WHERE eventTime > date_format(date_add('day', -30, current_date), '%Y-%m-%dT%H:%i:%sZ')
GROUP BY userIdentity.arn
ORDER BY api_call_count DESC
LIMIT 50;
-- Failed API calls indicating permission issues or reconnaissance
SELECT eventTime, userIdentity.arn, eventName, errorCode, errorMessage,
sourceIPAddress
FROM cloudtrail_logs
WHERE errorCode IN ('AccessDenied', 'UnauthorizedAccess', 'Client.UnauthorizedAccess')
AND eventTime > date_format(date_add('day', -7, current_date), '%Y-%m-%dT%H:%i:%sZ')
ORDER BY eventTime DESC
LIMIT 200;
-- Track KMS key usage
SELECT eventTime, userIdentity.arn, eventName, requestParameters,
resources[1].arn AS key_arn
FROM cloudtrail_logs
WHERE eventSource = 'kms.amazonaws.com'
AND eventName IN ('Decrypt', 'Encrypt', 'GenerateDataKey', 'DisableKey', 'ScheduleKeyDeletion')
AND eventTime > date_format(date_add('day', -7, current_date), '%Y-%m-%dT%H:%i:%sZ')
ORDER BY eventTime DESC;
```
## CloudTrail Lake (Event Data Store)
```bash
# Create an event data store for long-term queryable storage
aws cloudtrail create-event-data-store \
--name org-audit-event-store \
--multi-region-enabled \
--organization-enabled \
--retention-period 2555 \
--advanced-event-selectors '[
{
"Name": "AllManagementEvents",
"FieldSelectors": [
{"Field": "eventCategory", "Equals": ["Management"]}
]
}
]'
```
```sql
-- CloudTrail Lake SQL queries (run in console or via StartQuery API)
-- Investigate a specific user's activity
SELECT eventTime, eventName, eventSource, sourceIPAddress,
errorCode, requestParameters
FROM EVENT_DATA_STORE_ID
WHERE userIdentity.arn = 'arn:aws:iam::123456789012:user/suspicious-user'
AND eventTime > '2024-01-01 00:00:00'
ORDER BY eventTime DESC;
-- Cross-account activity summary
SELECT recipientAccountId, userIdentity.arn,
count(*) AS event_count
FROM EVENT_DATA_STORE_ID
WHERE eventTime > '2024-01-01 00:00:00'
GROUP BY recipientAccountId, userIdentity.arn
ORDER BY event_count DESC;
```
## Validate Trail Integrity
```bash
# Validate log file integrity for a date range
aws cloudtrail validate-logs \
--trail-arn arn:aws:cloudtrail:us-east-1:123456789012:trail/org-audit-trail \
--start-time "2024-01-01T00:00:00Z" \
--end-time "2024-01-31T23:59:59Z"
# Check trail status
aws cloudtrail get-trail-status --name org-audit-trail
# Describe the trail configuration
aws cloudtrail describe-trails --trail-name-list org-audit-trail
```
## Terraform Configuration
```hcl
resource "aws_cloudtrail" "org_trail" {
name = "org-audit-trail"
s3_bucket_name = aws_s3_bucket.cloudtrail.id
is_organization_trail = true
is_multi_region_trail = true
enable_log_file_validation = true
kms_key_id = aws_kms_key.cloudtrail.arn
cloud_watch_logs_group_arn = "${aws_cloudwatch_log_group.cloudtrail.arn}:*"
cloud_watch_logs_role_arn = aws_iam_role.cloudtrail_cw.arn
include_global_service_events = true
advanced_event_selector {
name = "AllManagementEvents"
field_selector {
field = "eventCategory"
equals = ["Management"]
}
}
advanced_event_selector {
name = "SensitiveS3DataEvents"
field_selector {
field = "eventCategory"
equals = ["Data"]
}
field_selector {
field = "resources.type"
equals = ["AWS::S3::Object"]
}
field_selector {
field = "resources.ARN"
starts_with = ["arn:aws:s3:::sensitive-data-bucket/"]
}
}
tags = {
Environment = "production"
Compliance = "soc2,hipaa"
}
}
```
## Setup Checklist
```yaml
cloudtrail_checklist:
trail_configuration:
- [ ] Organization trail enabled across all accounts
- [ ] Multi-region trail enabled
- [ ] Log file validation enabled
- [ ] KMS encryption configured with dedicated key
- [ ] CloudWatch Logs integration active
- [ ] S3 bucket policy restricts access to CloudTrail service only
s3_bucket_hardening:
- [ ] Public access blocked
- [ ] Versioning enabled
- [ ] Server-side encryption enabled
- [ ] Lifecycle policy set for retention and archival
- [ ] Access logging enabled on the bucket itself
- [ ] Object Lock enabled for WORM compliance (if required)
monitoring_and_alerting:
- [ ] Metric filters for unauthorized API calls
- [ ] Alarm on root account usage
- [ ] Alarm on console login without MFA
- [ ] Alarm on IAM policy changes
- [ ] Alarm on security group and NACL changes
- [ ] Alarm on CloudTrail configuration changes
- [ ] Alarm on S3 bucket policy changes
analysis:
- [ ] Athena table created for ad-hoc queries
- [ ] CloudTrail Lake event data store for long-term queries
- [ ] Regular review of high-risk API patterns
- [ ] Automated reports for compliance evidence
operational:
- [ ] Trail status health check automated
- [ ] Log delivery latency monitored
- [ ] Log file validation run periodically
- [ ] SNS notification for trail configuration changes
```
## Best Practices
- Organization-wide trails
- Enable log file validation
- Encrypt with KMS
- CloudWatch Logs integration
- Event alerting
- Enable organization-wide trails from the management account for full coverage
- Always enable log file validation to detect tampering
- Encrypt logs with a customer-managed KMS key and restrict key usage
- Use advanced event selectors to capture data events on sensitive resources without logging everything
- Integrate with CloudWatch Logs for real-time metric filters and alarms
- Set up Athena or CloudTrail Lake for efficient querying during investigations
- Apply S3 lifecycle policies to transition old logs to Glacier and enforce retention
- Monitor the trail itself (delivery errors, configuration changes) as a meta-control
- Validate log integrity periodically as part of compliance evidence collection
- Restrict access to the CloudTrail S3 bucket and KMS key with least-privilege IAM policies
compliance/auditing/azure-monitor-audit/SKILL.md
+322 -23
View File
@@ -9,49 +9,348 @@ metadata:
# Azure Monitor Audit
Audit Azure activity with Monitor and Activity Logs.
Audit Azure activity with Monitor, Activity Logs, and Log Analytics for compliance, security, and operational visibility.
## Diagnostic Settings
## When to Use
- Enabling centralized audit logging across Azure subscriptions
- Meeting compliance requirements for SOC 2, HIPAA, PCI DSS, or ISO 27001
- Investigating security incidents or unauthorized activity in Azure
- Setting up alerting on administrative and security events
- Building compliance dashboards and automated evidence collection
## Create Log Analytics Workspace
```bash
# Enable diagnostic settings
az monitor diagnostic-settings create \
--name audit-logs \
--resource /subscriptions/{sub}/resourceGroups/{rg}/providers/... \
--logs '[{"category":"AuditEvent","enabled":true}]' \
--workspace /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.OperationalInsights/workspaces/{workspace}
# Create resource group for audit resources
az group create \
--name rg-audit \
--location eastus
# Create Log Analytics workspace
az monitor log-analytics workspace create \
--resource-group rg-audit \
--workspace-name audit-workspace \
--location eastus \
--retention-time 365 \
--sku PerGB2018
# Get workspace ID for later use
WORKSPACE_ID=$(az monitor log-analytics workspace show \
--resource-group rg-audit \
--workspace-name audit-workspace \
--query id -o tsv)
# Enable audit solutions
az monitor log-analytics solution create \
--resource-group rg-audit \
--solution-type SecurityCenterFree \
--workspace audit-workspace
```
## Activity Log Export
## Configure Diagnostic Settings for Subscription Activity Log
```bash
# Export activity log to Log Analytics
# Export subscription activity log to Log Analytics
az monitor diagnostic-settings subscription create \
--name activity-log-export \
--name activity-log-to-workspace \
--location global \
--logs '[{"category":"Administrative","enabled":true},{"category":"Security","enabled":true}]' \
--workspace /subscriptions/.../workspaces/audit-workspace
--workspace "$WORKSPACE_ID" \
--logs '[
{"category": "Administrative", "enabled": true},
{"category": "Security", "enabled": true},
{"category": "ServiceHealth", "enabled": true},
{"category": "Alert", "enabled": true},
{"category": "Recommendation", "enabled": true},
{"category": "Policy", "enabled": true},
{"category": "Autoscale", "enabled": true},
{"category": "ResourceHealth", "enabled": true}
]'
# Also archive to storage account for long-term retention
az storage account create \
--name auditlogsarchive \
--resource-group rg-audit \
--location eastus \
--sku Standard_GRS \
--kind StorageV2 \
--min-tls-version TLS1_2 \
--allow-blob-public-access false
az monitor diagnostic-settings subscription create \
--name activity-log-to-storage \
--location global \
--storage-account /subscriptions/{sub}/resourceGroups/rg-audit/providers/Microsoft.Storage/storageAccounts/auditlogsarchive \
--logs '[
{"category": "Administrative", "enabled": true, "retentionPolicy": {"enabled": true, "days": 2555}},
{"category": "Security", "enabled": true, "retentionPolicy": {"enabled": true, "days": 2555}}
]'
```
## Log Analytics Queries
## Resource-Level Diagnostic Settings
```bash
# Enable diagnostics for Azure Key Vault
az monitor diagnostic-settings create \
--name keyvault-audit \
--resource /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.KeyVault/vaults/{vault} \
--workspace "$WORKSPACE_ID" \
--logs '[
{"category": "AuditEvent", "enabled": true, "retentionPolicy": {"enabled": true, "days": 365}},
{"category": "AzurePolicyEvaluationDetails", "enabled": true}
]' \
--metrics '[
{"category": "AllMetrics", "enabled": true}
]'
# Enable diagnostics for Azure SQL Database
az monitor diagnostic-settings create \
--name sql-audit \
--resource /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Sql/servers/{server}/databases/{db} \
--workspace "$WORKSPACE_ID" \
--logs '[
{"category": "SQLSecurityAuditEvents", "enabled": true},
{"category": "SQLInsights", "enabled": true},
{"category": "AutomaticTuning", "enabled": true}
]'
# Enable diagnostics for Azure App Service
az monitor diagnostic-settings create \
--name appservice-audit \
--resource /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Web/sites/{app} \
--workspace "$WORKSPACE_ID" \
--logs '[
{"category": "AppServiceHTTPLogs", "enabled": true},
{"category": "AppServiceAuditLogs", "enabled": true},
{"category": "AppServiceIPSecAuditLogs", "enabled": true},
{"category": "AppServicePlatformLogs", "enabled": true}
]'
# Enable diagnostics for Network Security Groups
az monitor diagnostic-settings create \
--name nsg-flow-logs \
--resource /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Network/networkSecurityGroups/{nsg} \
--workspace "$WORKSPACE_ID" \
--logs '[
{"category": "NetworkSecurityGroupEvent", "enabled": true},
{"category": "NetworkSecurityGroupRuleCounter", "enabled": true}
]'
```
## Azure Policy for Diagnostic Settings Enforcement
```bash
# Assign built-in policy to require diagnostic settings on Key Vaults
az policy assignment create \
--name require-kv-diagnostics \
--policy "951af2fa-529b-416e-ab6e-066fd85ac459" \
--scope /subscriptions/{sub} \
--params '{
"logAnalytics": {"value": "'$WORKSPACE_ID'"},
"effect": {"value": "DeployIfNotExists"}
}'
# Assign policy to require diagnostic settings on SQL databases
az policy assignment create \
--name require-sql-diagnostics \
--policy "b79fa14e-238a-4c2d-b376-442ce508fc84" \
--scope /subscriptions/{sub} \
--params '{
"logAnalyticsWorkspaceId": {"value": "'$WORKSPACE_ID'"}
}'
```
## KQL Queries for Security Investigation
```kusto
// Failed login attempts
AuditLogs
// Failed sign-in attempts with location and device details
SigninLogs
| where TimeGenerated > ago(24h)
| where ResultType != "0"
| project TimeGenerated, Identity, ResultDescription, IPAddress
| summarize FailureCount = count(),
DistinctIPs = dcount(IPAddress),
Locations = make_set(LocationDetails.city)
by UserPrincipalName, ResultDescription, AppDisplayName
| where FailureCount > 5
| order by FailureCount desc
// Administrative changes
// Successful sign-ins from unusual locations
SigninLogs
| where TimeGenerated > ago(7d)
| where ResultType == "0"
| extend City = tostring(LocationDetails.city),
Country = tostring(LocationDetails.countryOrRegion)
| summarize LoginCount = count(),
Cities = make_set(City),
Countries = make_set(Country)
by UserPrincipalName
| where array_length(Countries) > 2
// Risky sign-ins requiring investigation
SigninLogs
| where TimeGenerated > ago(7d)
| where RiskLevelDuringSignIn in ("medium", "high")
| project TimeGenerated, UserPrincipalName, IPAddress,
LocationDetails.city, RiskLevelDuringSignIn,
RiskEventTypes_V2, AppDisplayName
| order by TimeGenerated desc
// Administrative operations across subscriptions
AzureActivity
| where TimeGenerated > ago(24h)
| where CategoryValue == "Administrative"
| where OperationNameValue contains "write" or OperationNameValue contains "delete"
| project TimeGenerated, Caller, OperationNameValue, ResourceGroup
| where ActivityStatusValue == "Success"
| project TimeGenerated, Caller, OperationNameValue,
ResourceGroup, Resource, SubscriptionId
| order by TimeGenerated desc
// Key Vault access patterns
AzureDiagnostics
| where ResourceType == "VAULTS"
| where TimeGenerated > ago(24h)
| where OperationName in ("SecretGet", "SecretSet", "SecretDelete",
"KeySign", "KeyDecrypt", "CertificateGet")
| project TimeGenerated, CallerIPAddress, identity_claim_upn_s,
OperationName, id_s, ResultType
| order by TimeGenerated desc
// Detect changes to Network Security Groups
AzureActivity
| where TimeGenerated > ago(7d)
| where OperationNameValue has_any ("securityRules/write", "securityRules/delete",
"networkSecurityGroups/write")
| where ActivityStatusValue == "Success"
| project TimeGenerated, Caller, OperationNameValue,
ResourceGroup, Properties
| order by TimeGenerated desc
// Azure Policy compliance drift
PolicyInsights
| where TimeGenerated > ago(7d)
| where ComplianceState == "NonCompliant"
| summarize NonCompliantCount = count() by PolicyDefinitionName, ResourceType
| order by NonCompliantCount desc
// Privileged role assignments (PIM)
AuditLogs
| where TimeGenerated > ago(30d)
| where OperationName has_any ("Add member to role", "Add eligible member to role")
| extend RoleName = tostring(TargetResources[0].displayName),
AssignedUser = tostring(TargetResources[2].displayName),
AssignedBy = InitiatedBy.user.userPrincipalName
| project TimeGenerated, AssignedBy, AssignedUser, RoleName, OperationName
| order by TimeGenerated desc
```
## Alert Rules
```bash
# Create action group for security notifications
az monitor action-group create \
--resource-group rg-audit \
--name security-team \
--short-name SecTeam \
--email-receivers name=SecurityLead email=security@example.com \
--webhook-receivers name=PagerDuty uri=https://events.pagerduty.com/integration/{key}/enqueue
# Alert on multiple failed sign-ins (brute force detection)
az monitor scheduled-query create \
--resource-group rg-audit \
--name brute-force-detection \
--scopes "$WORKSPACE_ID" \
--condition "count > 10" \
--condition-query "SigninLogs | where ResultType != '0' | summarize count() by UserPrincipalName, bin(TimeGenerated, 5m) | where count_ > 10" \
--evaluation-frequency 5m \
--window-size 5m \
--severity 2 \
--action-groups /subscriptions/{sub}/resourceGroups/rg-audit/providers/Microsoft.Insights/actionGroups/security-team
# Alert on Key Vault secret access outside business hours
az monitor scheduled-query create \
--resource-group rg-audit \
--name keyvault-offhours-access \
--scopes "$WORKSPACE_ID" \
--condition "count > 0" \
--condition-query "AzureDiagnostics | where ResourceType == 'VAULTS' | where OperationName in ('SecretGet','SecretList') | where hourofday(TimeGenerated) < 6 or hourofday(TimeGenerated) > 22" \
--evaluation-frequency 15m \
--window-size 15m \
--severity 3 \
--action-groups /subscriptions/{sub}/resourceGroups/rg-audit/providers/Microsoft.Insights/actionGroups/security-team
# Alert on subscription-level administrative changes
az monitor activity-log alert create \
--resource-group rg-audit \
--name critical-admin-changes \
--condition category=Administrative and operationName="Microsoft.Authorization/roleAssignments/write" \
--action-group /subscriptions/{sub}/resourceGroups/rg-audit/providers/Microsoft.Insights/actionGroups/security-team \
--description "Alert on new role assignments"
```
## Workbook for Compliance Dashboard (ARM Template Snippet)
```json
{
"type": "Microsoft.Insights/workbooks",
"apiVersion": "2022-04-01",
"name": "[guid('compliance-dashboard')]",
"location": "[resourceGroup().location]",
"kind": "shared",
"properties": {
"displayName": "Compliance Audit Dashboard",
"serializedData": "{\"version\":\"Notebook/1.0\",\"items\":[{\"type\":1,\"content\":{\"json\":\"## Compliance Audit Dashboard\"},\"name\":\"title\"},{\"type\":3,\"content\":{\"version\":\"KqlItem/1.0\",\"query\":\"SigninLogs | where TimeGenerated > ago(24h) | where ResultType != '0' | summarize count() by bin(TimeGenerated, 1h)\",\"size\":0,\"title\":\"Failed Sign-ins (24h)\",\"timeContext\":{\"durationMs\":86400000},\"queryType\":0},\"name\":\"failed-signins\"}]}"
}
}
```
## Setup Checklist
```yaml
azure_monitor_checklist:
workspace_setup:
- [ ] Log Analytics workspace created in appropriate region
- [ ] Retention period configured (minimum per compliance framework)
- [ ] Daily cap configured to prevent cost overruns
- [ ] RBAC permissions set (Log Analytics Reader for auditors)
diagnostic_settings:
- [ ] Subscription activity log exported to Log Analytics
- [ ] Subscription activity log archived to storage account
- [ ] Key Vault audit events enabled
- [ ] Azure SQL audit logging enabled
- [ ] NSG flow logs enabled
- [ ] App Service audit logs enabled
- [ ] Azure AD sign-in and audit logs connected
policy_enforcement:
- [ ] Azure Policy assigned to enforce diagnostic settings
- [ ] DeployIfNotExists policies for critical resource types
- [ ] Compliance state monitored via Policy Insights
alerting:
- [ ] Action groups configured for security and operations teams
- [ ] Alert on brute force sign-in attempts
- [ ] Alert on privileged role assignments
- [ ] Alert on Key Vault sensitive operations
- [ ] Alert on NSG rule changes
- [ ] Alert on resource deletions in production
reporting:
- [ ] Compliance workbook deployed
- [ ] Weekly automated query reports exported
- [ ] Quarterly access review queries prepared
- [ ] Evidence collection queries documented for auditors
```
## Best Practices
- Centralize to Log Analytics
- Long-term archive to Storage
- Configure alerts
- Regular query reviews
- Centralize all audit data into a single Log Analytics workspace per tenant
- Archive logs to immutable storage for long-term retention and compliance
- Use Azure Policy with DeployIfNotExists to enforce diagnostic settings on new resources
- Create saved KQL queries for common investigation and compliance scenarios
- Set up scheduled query alerts for security-critical events
- Assign Log Analytics Reader role to auditors without granting broader access
- Monitor the diagnostic settings pipeline itself for delivery failures
- Use workbooks for visual compliance dashboards shared with stakeholders
- Export query results on a schedule for compliance evidence packages
- Separate operational and security alerting to avoid alert fatigue
compliance/auditing/gcp-audit-logs/SKILL.md
+437 -34
View File
@@ -9,63 +9,466 @@ metadata:
# GCP Audit Logs
Audit GCP activity with Cloud Audit Logs.
Audit GCP activity with Cloud Audit Logs for compliance, security investigation, and operational monitoring.
## When to Use
- Enabling organization-wide audit logging across GCP projects
- Meeting compliance requirements for SOC 2, HIPAA, PCI DSS, or FedRAMP
- Investigating unauthorized access or suspicious API activity
- Setting up alerting on administrative and data access events
- Exporting logs to BigQuery for long-term analysis and reporting
## Audit Log Types
```yaml
log_types:
admin_activity:
- Always enabled
- API calls that modify resources
- No charge
description: API calls that modify resource configuration or metadata
enabled: Always (cannot be disabled)
retention: 400 days (default)
cost: No charge
examples:
- Creating or deleting VM instances
- Changing IAM policies
- Modifying firewall rules
data_access:
- Must be enabled
- Read/write data operations
- Can be high volume
description: API calls that read resource configuration, metadata, or user data
enabled: Must be explicitly enabled (except BigQuery)
retention: 30 days (default)
cost: Can be significant at high volume
subtypes:
ADMIN_READ: Read resource configuration/metadata
DATA_READ: Read user-provided data
DATA_WRITE: Write user-provided data
system_event:
- Always enabled
- GCP system actions
description: Actions performed by GCP systems on behalf of resources
enabled: Always (cannot be disabled)
retention: 400 days (default)
cost: No charge
examples:
- Live migration of VM instances
- Automatic scaling events
policy_denied:
- Always enabled
- Access denials
description: Actions denied by VPC Service Controls or organization policies
enabled: Always (cannot be disabled)
retention: 400 days (default)
cost: No charge
```
## Enable Data Access Logs
## Enable Data Access Logs for an Organization
```bash
# Enable for all services
gcloud logging sinks create audit-sink \
storage.googleapis.com/audit-logs-bucket \
--log-filter='logName:"cloudaudit.googleapis.com"'
# Get current org IAM policy
gcloud organizations get-iam-policy ORG_ID --format=json > org-policy.json
# IAM policy for data access logs
gcloud projects get-iam-policy PROJECT_ID > policy.yaml
# Add auditConfigs section
gcloud projects set-iam-policy PROJECT_ID policy.yaml
# Add audit config to org-policy.json:
# {
# "auditConfigs": [
# {
# "service": "allServices",
# "auditLogConfigs": [
# {"logType": "ADMIN_READ"},
# {"logType": "DATA_READ"},
# {"logType": "DATA_WRITE"}
# ]
# }
# ],
# ...existing bindings...
# }
# Apply the updated policy
gcloud organizations set-iam-policy ORG_ID org-policy.json
# Enable data access logs for specific services at project level
gcloud projects get-iam-policy PROJECT_ID --format=json > project-policy.json
# Example: enable only for Cloud Storage and BigQuery
# {
# "auditConfigs": [
# {
# "service": "storage.googleapis.com",
# "auditLogConfigs": [
# {"logType": "DATA_READ"},
# {"logType": "DATA_WRITE"}
# ]
# },
# {
# "service": "bigquery.googleapis.com",
# "auditLogConfigs": [
# {"logType": "DATA_READ"},
# {"logType": "DATA_WRITE"}
# ]
# }
# ]
# }
gcloud projects set-iam-policy PROJECT_ID project-policy.json
```
## BigQuery Analysis
## Configure Log Sinks for Export
```bash
# Create BigQuery dataset for audit log export
bq mk --dataset \
--description "Audit log export" \
--default_table_expiration 0 \
--location US \
PROJECT_ID:audit_logs
# Create organization-level log sink to BigQuery
gcloud logging sinks create org-audit-bigquery \
bigquery.googleapis.com/projects/PROJECT_ID/datasets/audit_logs \
--organization=ORG_ID \
--include-children \
--log-filter='logName:"cloudaudit.googleapis.com"'
# Get the sink writer identity and grant BigQuery access
SINK_SA=$(gcloud logging sinks describe org-audit-bigquery \
--organization=ORG_ID --format='value(writerIdentity)')
bq add-iam-policy-binding \
--member="$SINK_SA" \
--role="roles/bigquery.dataEditor" \
PROJECT_ID:audit_logs
# Create Cloud Storage sink for long-term archive
gsutil mb -l US -b on gs://org-audit-logs-archive
gsutil retention set 7y gs://org-audit-logs-archive
gcloud logging sinks create org-audit-storage \
storage.googleapis.com/org-audit-logs-archive \
--organization=ORG_ID \
--include-children \
--log-filter='logName:"cloudaudit.googleapis.com"'
STORAGE_SA=$(gcloud logging sinks describe org-audit-storage \
--organization=ORG_ID --format='value(writerIdentity)')
gsutil iam ch "$STORAGE_SA:objectCreator" gs://org-audit-logs-archive
# Create Pub/Sub sink for real-time streaming to SIEM
gcloud pubsub topics create audit-log-stream
gcloud logging sinks create org-audit-pubsub \
pubsub.googleapis.com/projects/PROJECT_ID/topics/audit-log-stream \
--organization=ORG_ID \
--include-children \
--log-filter='logName:"cloudaudit.googleapis.com" AND (protoPayload.methodName:"delete" OR protoPayload.methodName:"setIamPolicy" OR severity>=WARNING)'
PUBSUB_SA=$(gcloud logging sinks describe org-audit-pubsub \
--organization=ORG_ID --format='value(writerIdentity)')
gcloud pubsub topics add-iam-policy-binding audit-log-stream \
--member="$PUBSUB_SA" \
--role="roles/pubsub.publisher"
```
## Logging Queries (Cloud Logging Explorer)
```bash
# View admin activity logs for the last 24 hours
gcloud logging read 'logName:"cloudaudit.googleapis.com/activity"
AND timestamp>="2024-01-01T00:00:00Z"' \
--project=PROJECT_ID \
--format=json \
--limit=100
# Find IAM policy changes
gcloud logging read 'logName:"cloudaudit.googleapis.com/activity"
AND protoPayload.methodName="SetIamPolicy"' \
--project=PROJECT_ID \
--freshness=7d
# Find resource deletions
gcloud logging read 'logName:"cloudaudit.googleapis.com/activity"
AND protoPayload.methodName=~"delete"
AND severity>=NOTICE' \
--project=PROJECT_ID \
--freshness=7d
# Data access audit log entries
gcloud logging read 'logName:"cloudaudit.googleapis.com/data_access"
AND protoPayload.serviceName="storage.googleapis.com"
AND protoPayload.methodName="storage.objects.get"' \
--project=PROJECT_ID \
--freshness=24h
# Failed authorization attempts
gcloud logging read 'logName:"cloudaudit.googleapis.com/policy"' \
--project=PROJECT_ID \
--freshness=7d
```
## BigQuery Analysis Queries
```sql
-- Query audit logs from BigQuery export
-- All destructive operations in the last 30 days
SELECT
timestamp,
protopayload_auditlog.authenticationInfo.principalEmail,
protopayload_auditlog.methodName,
resource.labels.project_id
FROM `project.dataset.cloudaudit_googleapis_com_activity_*`
WHERE timestamp > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
protopayload_auditlog.authenticationInfo.principalEmail AS principal,
protopayload_auditlog.methodName AS method,
protopayload_auditlog.resourceName AS resource,
resource.labels.project_id AS project,
protopayload_auditlog.status.code AS status_code,
protopayload_auditlog.status.message AS status_message
FROM `project.audit_logs.cloudaudit_googleapis_com_activity_*`
WHERE _TABLE_SUFFIX >= FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY))
AND protopayload_auditlog.methodName LIKE '%delete%'
ORDER BY timestamp DESC
LIMIT 500;
-- IAM policy changes across the organization
SELECT
timestamp,
protopayload_auditlog.authenticationInfo.principalEmail AS changed_by,
resource.labels.project_id AS project,
protopayload_auditlog.resourceName AS resource,
protopayload_auditlog.servicedata_v1_iam.policyDelta.bindingDeltas
FROM `project.audit_logs.cloudaudit_googleapis_com_activity_*`
WHERE _TABLE_SUFFIX >= FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY))
AND protopayload_auditlog.methodName = 'SetIamPolicy'
ORDER BY timestamp DESC;
-- Activity per principal (detect anomalous usage)
SELECT
protopayload_auditlog.authenticationInfo.principalEmail AS principal,
COUNT(*) AS action_count,
COUNT(DISTINCT protopayload_auditlog.methodName) AS unique_methods,
COUNT(DISTINCT protopayload_auditlog.requestMetadata.callerIp) AS unique_ips,
MIN(timestamp) AS first_activity,
MAX(timestamp) AS last_activity
FROM `project.audit_logs.cloudaudit_googleapis_com_activity_*`
WHERE _TABLE_SUFFIX >= FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY))
GROUP BY principal
ORDER BY action_count DESC
LIMIT 50;
-- Service account key creation events (security risk indicator)
SELECT
timestamp,
protopayload_auditlog.authenticationInfo.principalEmail AS created_by,
protopayload_auditlog.resourceName AS service_account,
protopayload_auditlog.requestMetadata.callerIp AS source_ip
FROM `project.audit_logs.cloudaudit_googleapis_com_activity_*`
WHERE _TABLE_SUFFIX >= FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY))
AND protopayload_auditlog.methodName = 'google.iam.admin.v1.CreateServiceAccountKey'
ORDER BY timestamp DESC;
-- Data access patterns for sensitive Cloud Storage buckets
SELECT
timestamp,
protopayload_auditlog.authenticationInfo.principalEmail AS accessor,
protopayload_auditlog.resourceName AS object_path,
protopayload_auditlog.methodName AS access_type,
protopayload_auditlog.requestMetadata.callerIp AS source_ip
FROM `project.audit_logs.cloudaudit_googleapis_com_data_access_*`
WHERE _TABLE_SUFFIX >= FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY))
AND protopayload_auditlog.resourceName LIKE '%sensitive-bucket%'
ORDER BY timestamp DESC
LIMIT 1000;
-- Failed operations indicating permission issues
SELECT
timestamp,
protopayload_auditlog.authenticationInfo.principalEmail AS principal,
protopayload_auditlog.methodName AS method,
protopayload_auditlog.status.code AS error_code,
protopayload_auditlog.status.message AS error_message,
protopayload_auditlog.requestMetadata.callerIp AS source_ip
FROM `project.audit_logs.cloudaudit_googleapis_com_activity_*`
WHERE _TABLE_SUFFIX >= FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY))
AND protopayload_auditlog.status.code != 0
ORDER BY timestamp DESC
LIMIT 500;
```
## Alerting Policies
```bash
# Alert on service account key creation
gcloud alpha monitoring policies create \
--display-name="SA Key Created" \
--condition-display-name="Service Account Key Creation" \
--condition-filter='resource.type="audited_resource" AND protoPayload.methodName="google.iam.admin.v1.CreateServiceAccountKey"' \
--condition-threshold-value=0 \
--condition-threshold-comparison=COMPARISON_GT \
--condition-threshold-duration=0s \
--notification-channels=projects/PROJECT_ID/notificationChannels/CHANNEL_ID \
--combiner=OR
# Create a log-based metric for IAM changes
gcloud logging metrics create iam-policy-changes \
--description="Count of IAM policy changes" \
--log-filter='logName:"cloudaudit.googleapis.com/activity" AND protoPayload.methodName="SetIamPolicy"'
# Create alerting policy using the log-based metric
gcloud alpha monitoring policies create \
--display-name="IAM Policy Changes" \
--condition-display-name="IAM Changes Detected" \
--condition-filter='metric.type="logging.googleapis.com/user/iam-policy-changes"' \
--condition-threshold-value=0 \
--condition-threshold-comparison=COMPARISON_GT \
--condition-threshold-duration=0s \
--notification-channels=projects/PROJECT_ID/notificationChannels/CHANNEL_ID
# Create log-based metric for firewall changes
gcloud logging metrics create firewall-rule-changes \
--description="Count of firewall rule changes" \
--log-filter='logName:"cloudaudit.googleapis.com/activity"
AND (protoPayload.methodName="v1.compute.firewalls.insert"
OR protoPayload.methodName="v1.compute.firewalls.delete"
OR protoPayload.methodName="v1.compute.firewalls.patch")'
# Create log-based metric for VPC network changes
gcloud logging metrics create vpc-network-changes \
--description="Count of VPC network changes" \
--log-filter='logName:"cloudaudit.googleapis.com/activity"
AND resource.type="gce_network"
AND (protoPayload.methodName=~"insert$" OR protoPayload.methodName=~"delete$")'
```
## Terraform Configuration
```hcl
# Organization-level audit log sink to BigQuery
resource "google_logging_organization_sink" "audit_bigquery" {
name = "org-audit-bigquery"
org_id = var.org_id
destination = "bigquery.googleapis.com/projects/${var.project_id}/datasets/${google_bigquery_dataset.audit_logs.dataset_id}"
filter = "logName:\"cloudaudit.googleapis.com\""
include_children = true
bigquery_options {
use_partitioned_tables = true
}
}
resource "google_bigquery_dataset" "audit_logs" {
dataset_id = "audit_logs"
project = var.project_id
location = "US"
description = "Organization audit log export"
default_table_expiration_ms = null # No auto-expiry
access {
role = "WRITER"
user_by_email = google_logging_organization_sink.audit_bigquery.writer_identity
}
access {
role = "READER"
group_by_email = "security-auditors@example.com"
}
}
# Retention bucket with bucket lock
resource "google_storage_bucket" "audit_archive" {
name = "org-audit-logs-archive"
location = "US"
force_destroy = false
project = var.project_id
uniform_bucket_level_access = true
retention_policy {
is_locked = true
retention_period = 220752000 # 7 years in seconds
}
lifecycle_rule {
condition {
age = 90
}
action {
type = "SetStorageClass"
storage_class = "COLDLINE"
}
}
}
# Log-based alerting
resource "google_logging_metric" "iam_changes" {
name = "iam-policy-changes"
project = var.project_id
filter = "logName:\"cloudaudit.googleapis.com/activity\" AND protoPayload.methodName=\"SetIamPolicy\""
metric_descriptor {
metric_kind = "DELTA"
value_type = "INT64"
}
}
resource "google_monitoring_alert_policy" "iam_changes" {
display_name = "IAM Policy Changes Detected"
project = var.project_id
combiner = "OR"
conditions {
display_name = "IAM policy change count"
condition_threshold {
filter = "metric.type=\"logging.googleapis.com/user/iam-policy-changes\" AND resource.type=\"global\""
comparison = "COMPARISON_GT"
threshold_value = 0
duration = "0s"
}
}
notification_channels = [var.notification_channel_id]
}
```
## Setup Checklist
```yaml
gcp_audit_logs_checklist:
log_enablement:
- [ ] Admin activity logs verified active (always on)
- [ ] Data access logs enabled for sensitive services
- [ ] Data access exemptions configured to exclude high-volume, low-risk operations
- [ ] System event logs verified active (always on)
log_routing:
- [ ] Organization-level sink to BigQuery for analysis
- [ ] Organization-level sink to Cloud Storage for long-term archive
- [ ] Pub/Sub sink for real-time SIEM streaming (high severity events)
- [ ] Sink writer identities granted appropriate destination permissions
- [ ] Inclusion filters verified to capture all audit log types
storage_and_retention:
- [ ] BigQuery dataset created with appropriate access controls
- [ ] Cloud Storage bucket with retention policy and bucket lock
- [ ] Storage class lifecycle rules configured (Standard to Coldline)
- [ ] Default log retention in Cloud Logging extended if needed
alerting:
- [ ] Notification channels configured (email, PagerDuty, Slack)
- [ ] Log-based metric for IAM policy changes
- [ ] Log-based metric for firewall rule changes
- [ ] Log-based metric for service account key creation
- [ ] Alert policy for each critical metric
- [ ] Alert notification tested end-to-end
access_control:
- [ ] Logging Admin role restricted to security team
- [ ] BigQuery dataset read access granted to auditors only
- [ ] Storage bucket access restricted with IAM
- [ ] Sink configuration changes monitored via admin activity logs
```
## Best Practices
- Export to BigQuery for analysis
- Configure log retention
- Enable data access logs for sensitive resources
- Set up alerting policies
- Enable data access logs selectively on sensitive services to control cost and volume
- Use organization-level sinks with include-children to capture all projects automatically
- Export to BigQuery with partitioned tables for efficient querying over large time ranges
- Archive to Cloud Storage with bucket lock and retention policies for immutable long-term storage
- Create log-based metrics and alerting policies for high-severity events
- Stream critical audit events via Pub/Sub to SIEM for real-time correlation
- Apply exemptions to exclude high-volume read-only service accounts from data access logs
- Restrict access to audit log sinks and destinations with least-privilege IAM bindings
- Regularly run BigQuery analysis queries to detect anomalous patterns and generate compliance reports
- Monitor log sink health and delivery latency to ensure continuous audit coverage
compliance/continuity/business-continuity/SKILL.md
+411 -57
View File
@@ -9,78 +9,432 @@ metadata:
# Business Continuity Planning
Develop and maintain business continuity capabilities.
Develop and maintain business continuity capabilities including Business Impact Analysis, communication plans, recovery procedures, and testing schedules for organizational resilience.
## When to Use
- Developing a formal Business Continuity Plan (BCP) for the organization
- Conducting a Business Impact Analysis (BIA) to prioritize recovery efforts
- Establishing communication plans for crisis scenarios
- Defining recovery procedures for critical business processes
- Scheduling and conducting BCP exercises and tests
- Meeting compliance requirements for continuity planning (SOC 2, ISO 27001, HIPAA, FedRAMP)
## BCP Framework
```yaml
bcp_phases:
1_analysis:
- Business Impact Analysis (BIA)
- Risk assessment
- Critical process identification
2_planning:
- Recovery strategies
- Resource requirements
- Communication plans
3_implementation:
- Procedure documentation
- Training
- Technology setup
4_testing:
- Plan exercises
- Gap identification
- Continuous improvement
1_governance:
actions:
- Obtain executive sponsorship and funding
- Assign BCP coordinator and team
- Define BCP scope and policy
- Establish BCP committee with cross-functional representation
deliverables:
- BCP policy statement
- BCP team charter and roster
- Scope document
2_analysis:
actions:
- Conduct Business Impact Analysis (BIA)
- Perform risk assessment for continuity threats
- Identify critical business processes and dependencies
- Determine recovery priorities and resource requirements
deliverables:
- BIA report
- Risk assessment report
- Critical process inventory
3_strategy:
actions:
- Select recovery strategies for each critical process
- Identify alternate work arrangements (remote, alternate site)
- Define technology recovery strategies (DR plan)
- Establish vendor and supply chain contingencies
deliverables:
- Recovery strategy document
- Technology recovery plan
- Alternate site arrangements
4_plan_development:
actions:
- Write detailed recovery procedures
- Develop communication plans (internal and external)
- Create emergency response procedures
- Document roles, responsibilities, and contact information
deliverables:
- Business Continuity Plan document
- Communication plan
- Emergency response procedures
- Contact lists and call trees
5_testing:
actions:
- Develop test plan and schedule
- Conduct exercises (tabletop, functional, full-scale)
- Evaluate results and identify gaps
- Update plans based on lessons learned
deliverables:
- Test plan
- Exercise reports
- Updated BCP based on findings
6_maintenance:
actions:
- Review and update BCP annually (minimum)
- Update after significant organizational changes
- Refresh BIA when business processes change
- Maintain training and awareness program
deliverables:
- Annual BCP review record
- Updated BIA (if changes occurred)
- Training completion records
```
## Business Impact Analysis
## Business Impact Analysis Template
```yaml
process_classification:
critical:
max_downtime: 4 hours
examples: Payment processing, authentication
essential:
max_downtime: 24 hours
examples: Customer support, reporting
necessary:
max_downtime: 72 hours
examples: Internal tools, analytics
desirable:
max_downtime: 7 days
examples: Development environments
bia_template:
process_assessment:
process_name: ""
process_owner: ""
department: ""
description: ""
criticality_classification:
mission_critical:
max_tolerable_downtime: "0-4 hours"
description: "Failure causes immediate, severe impact to customers or revenue"
examples:
- Payment processing
- Authentication and authorization
- Core API serving customer requests
- Order fulfillment
essential:
max_tolerable_downtime: "4-24 hours"
description: "Failure causes significant degradation but not complete loss"
examples:
- Customer support systems
- Reporting and dashboards
- Email and notifications
- Billing and invoicing
important:
max_tolerable_downtime: "1-3 days"
description: "Failure causes inconvenience and workarounds are available"
examples:
- Internal collaboration tools
- Analytics and BI platforms
- HR self-service systems
- Knowledge base
non_essential:
max_tolerable_downtime: "3-7 days"
description: "Failure has minimal operational impact"
examples:
- Development and test environments
- Training platforms
- Archive systems
impact_categories:
financial:
revenue_loss_per_hour: ""
penalty_or_fine_risk: ""
recovery_cost_estimate: ""
operational:
affected_employees: ""
affected_customers: ""
workaround_available: "yes/no"
workaround_description: ""
reputational:
customer_visibility: "high/medium/low"
media_attention_risk: "high/medium/low"
regulatory_reporting_required: "yes/no"
legal_regulatory:
compliance_impact: ""
contractual_sla_breach: "yes/no"
sla_penalty_details: ""
dependencies:
technology:
- system: ""
rto: ""
rpo: ""
dr_strategy: ""
people:
- role: ""
minimum_staff: ""
remote_capable: "yes/no"
vendors:
- vendor: ""
service: ""
sla: ""
alternative: ""
facilities:
- location: ""
alternative: ""
recovery_requirements:
rto: ""
rpo: ""
minimum_recovery_level: "Description of minimum acceptable service"
full_recovery_target: "Time to full normal operations"
```
## Communication Plan
```yaml
communication:
internal:
- Executive notification
- Team communication
- Status updates
external:
- Customer notification
- Regulatory reporting
- Media relations
channels:
- Primary: Slack/Teams
- Secondary: Email
- Emergency: Phone tree
communication_plan:
activation_criteria:
- Event affecting multiple critical systems
- Physical facility unavailable
- Pandemic or workforce availability crisis
- Major vendor/partner outage
- Cybersecurity incident with operational impact
internal_communication:
executive_notification:
who: "CEO, CTO, CFO, VP Engineering, VP Operations"
when: "Within 15 minutes of BCP activation"
method: "Phone call (primary), SMS (secondary)"
message_template: |
BUSINESS CONTINUITY EVENT ACTIVATED
Incident: [Brief description]
Impact: [Systems/processes affected]
Status: [Current state]
Next update: [Time]
Bridge call: [Number/link]
team_notification:
who: "All affected department leads and their teams"
when: "Within 30 minutes of BCP activation"
method: "Slack/Teams (primary), Email (secondary), SMS (tertiary)"
message_template: |
BCP ACTIVATED - [Event Type]
What happened: [Description]
What is affected: [Systems/services]
What to do: [Immediate actions for your team]
Status updates: [Channel/frequency]
Questions: Contact [BCP coordinator]
all_staff_notification:
who: "All employees"
when: "Within 1 hour of BCP activation"
method: "Email, Slack/Teams announcement, intranet"
content: "Situation summary, impact on work, expectations"
status_updates:
frequency: "Every 2 hours during active event, daily after stabilization"
channel: "Dedicated Slack channel, email distribution list"
content: "Current status, actions taken, next steps, timeline"
external_communication:
customers:
who: "Affected customers"
when: "Within 2 hours of BCP activation (if customer-facing impact)"
method: "Status page update, email, in-app notification"
message_template: |
We are currently experiencing [issue description].
Impact: [What customers may notice]
Status: We are actively working to resolve this.
Updates: Follow our status page at status.example.com
ETA: [Estimated resolution time or "investigating"]
regulatory:
who: "Applicable regulatory bodies"
when: "Per regulatory requirements (e.g., 72 hours for GDPR breach)"
method: "Formal notification per regulatory procedure"
media:
who: "Press inquiries"
when: "Only if media attention occurs"
method: "Prepared statement through communications team"
rule: "All media inquiries routed to designated spokesperson"
vendors_partners:
who: "Critical vendors and business partners"
when: "Within 4 hours if partner services affected"
method: "Direct contact via relationship manager"
contact_lists:
maintenance: "Updated quarterly"
storage: "Accessible offline (printed, mobile app, cloud-independent)"
includes:
- BCP team members (name, role, phone, email, alternate phone)
- Executive team
- Department leads
- Key vendor contacts
- Regulatory contacts
- Legal counsel
- Insurance broker
- PR/communications firm
```
## Recovery Procedures
```yaml
recovery_procedures:
immediate_response:
step_1: "Incident commander assesses situation and declares BCP activation"
step_2: "Notify BCP team and establish command structure"
step_3: "Activate communication plan"
step_4: "Assess damage and determine scope of disruption"
step_5: "Initiate appropriate recovery procedures based on scenario"
scenario_specific:
data_center_or_region_outage:
- Activate DR failover procedures
- Redirect traffic to DR region
- Verify service restoration
- Communicate status to stakeholders
- Plan return to primary when available
cybersecurity_incident:
- Engage incident response team
- Contain the threat (isolate affected systems)
- Assess data impact and potential breach
- Activate forensic investigation
- Restore from known-good backups if needed
- Notify legal and regulatory as required
pandemic_workforce_disruption:
- Activate remote work procedures
- Verify VPN and remote access capacity
- Redistribute critical functions if staff unavailable
- Implement shift rotations to maintain coverage
- Assess vendor ability to maintain service levels
key_vendor_failure:
- Assess impact on dependent business processes
- Activate vendor contingency plan
- Engage alternate vendor if available
- Implement manual workarounds as needed
- Communicate impact to affected stakeholders
facility_unavailable:
- Account for all personnel safety
- Activate alternate work site arrangements
- Redirect mail and deliveries
- Set up temporary communication channels
- Assess timeline for facility restoration
stabilization:
- Monitor recovered services continuously
- Address any residual issues
- Begin planning return to normal operations
- Continue stakeholder communication
- Document all actions and decisions
return_to_normal:
- Develop return-to-normal plan
- Execute failback procedures (if DR was activated)
- Verify data consistency and integrity
- Restore standard operating procedures
- Conduct post-event review
- Update BCP based on lessons learned
```
## Testing Schedule and Types
```yaml
testing_schedule:
tabletop_exercise:
frequency: "Quarterly"
duration: "2-3 hours"
participants: "BCP team, department leads, executive sponsor"
format: "Facilitated discussion of a scenario"
scenarios_to_rotate:
- Major cloud provider region outage
- Ransomware attack on production systems
- Key employee unavailability (bus factor scenario)
- Critical vendor goes out of business
- Office building inaccessible
output: "Exercise report with findings and action items"
functional_exercise:
frequency: "Semi-annually"
duration: "4-8 hours"
participants: "BCP team, IT operations, affected departments"
format: "Execute specific recovery procedures without full disruption"
examples:
- "Activate remote work for one department for a day"
- "Failover a non-production database and verify application connectivity"
- "Execute communication plan and verify contact list accuracy"
- "Restore a critical system from backup in an isolated environment"
output: "Functional test report with measured recovery times"
full_scale_exercise:
frequency: "Annually"
duration: "1-2 days"
participants: "All BCP team members, IT, communications, management"
format: "Simulate a major disruption and execute full recovery"
includes:
- "Activate BCP command structure"
- "Execute DR failover for production systems"
- "Activate communication plan"
- "Operate from alternate arrangements for set period"
- "Execute failback and return to normal"
output: "Full exercise report with comprehensive metrics and lessons learned"
testing_metrics:
- "Time to activate BCP command structure"
- "Time to complete communication notifications"
- "Contact list accuracy (% reachable)"
- "Actual RTO vs. target RTO per system"
- "Actual RPO vs. target RPO per system"
- "Number of issues identified"
- "Number of runbook corrections needed"
```
## BCP Maintenance Checklist
```yaml
bcp_maintenance_checklist:
quarterly:
- [ ] Contact lists verified and updated
- [ ] Tabletop exercise conducted
- [ ] BCP team roster reviewed
- [ ] Vendor contact information verified
- [ ] Communication channels tested
semi_annually:
- [ ] Functional exercise conducted
- [ ] Recovery procedures reviewed for accuracy
- [ ] Technology dependencies verified
- [ ] Vendor continuity capabilities confirmed
annually:
- [ ] Full-scale exercise conducted
- [ ] Business Impact Analysis refreshed
- [ ] Risk assessment updated
- [ ] BCP document fully reviewed and updated
- [ ] Executive review and sign-off obtained
- [ ] Training completed for all BCP team members
- [ ] Lessons learned from all exercises incorporated
triggered_by_change:
- [ ] New critical business process added
- [ ] Major organizational restructuring
- [ ] Technology platform migration
- [ ] New regulatory requirement
- [ ] Significant vendor change
- [ ] Actual disruption event (post-event update)
```
## Best Practices
- Annual BIA updates
- Regular plan testing
- Clear roles and responsibilities
- Multiple communication channels
- Executive sponsorship
- Secure executive sponsorship: BCP without leadership commitment will not be taken seriously
- Base recovery priorities on Business Impact Analysis, not assumptions or technical preferences
- Test the communication plan independently: it fails more often than the technology recovery
- Maintain contact lists as if your primary systems are unavailable (offline copies, mobile access)
- Conduct tabletop exercises quarterly at minimum: they are low-cost and high-value for identifying gaps
- Include non-IT scenarios in planning (pandemic, facility loss, key personnel unavailability)
- Define clear activation criteria so there is no ambiguity about when to invoke the BCP
- Keep the BCP document practical and actionable, not a shelf document written for auditors
- Update the BCP after every significant organizational or technology change
- Review and incorporate lessons from every exercise and every real event into the plan
compliance/continuity/disaster-recovery/SKILL.md
+532 -50
View File
@@ -9,65 +9,547 @@ metadata:
# Disaster Recovery
Implement disaster recovery strategies and procedures.
Implement disaster recovery strategies including RTO/RPO planning, AWS cross-region failover patterns, DR testing procedures, and automated failover scripts.
## DR Metrics
## When to Use
- Defining RTO and RPO targets for critical systems
- Designing multi-region or multi-cloud disaster recovery architectures
- Implementing automated failover and failback procedures
- Conducting DR tests (tabletop, component, full failover)
- Meeting compliance requirements for contingency planning (SOC 2, HIPAA, FedRAMP, ISO 27001)
## RTO/RPO Planning
```yaml
recovery_metrics:
RTO: Recovery Time Objective
- Maximum acceptable downtime
- How long to restore service
RPO: Recovery Point Objective
- Maximum acceptable data loss
- How much data can be lost
RTO:
definition: "Recovery Time Objective - maximum acceptable downtime"
measurement: "From incident declaration to service restoration"
factors:
- Failover automation maturity
- Data replication lag
- DNS propagation time
- Application warm-up time
- Verification procedures
RPO:
definition: "Recovery Point Objective - maximum acceptable data loss"
measurement: "Time gap between last good backup and the incident"
factors:
- Backup frequency
- Replication method (sync vs. async)
- Transaction log shipping interval
- Cross-region replication lag
service_tier_targets:
tier_1_critical:
examples: "Authentication, payment processing, core API"
rto: "< 15 minutes"
rpo: "< 1 minute (near-zero)"
strategy: "Multi-site active-active or warm standby"
replication: "Synchronous or near-synchronous"
testing: "Quarterly failover test"
tier_2_essential:
examples: "Customer dashboards, reporting, notifications"
rto: "< 1 hour"
rpo: "< 15 minutes"
strategy: "Warm standby or pilot light"
replication: "Asynchronous with short interval"
testing: "Semi-annual failover test"
tier_3_standard:
examples: "Internal tools, analytics, batch processing"
rto: "< 4 hours"
rpo: "< 1 hour"
strategy: "Pilot light or backup and restore"
replication: "Periodic snapshots"
testing: "Annual failover test"
tier_4_non_essential:
examples: "Development environments, documentation sites"
rto: "< 24 hours"
rpo: "< 24 hours"
strategy: "Backup and restore"
replication: "Daily backups"
testing: "Annual backup restore verification"
```
## DR Strategies
| Strategy | RTO | RPO | Cost |
|----------|-----|-----|------|
| Backup & Restore | Hours | Hours | $ |
| Pilot Light | Minutes-Hours | Minutes | $$ |
| Warm Standby | Minutes | Seconds | $$$ |
| Multi-Site Active | Near-zero | Near-zero | $$$$ |
## AWS Multi-Region
```bash
# Cross-region RDS replica
aws rds create-db-instance-read-replica \
--db-instance-identifier dr-replica \
--source-db-instance-identifier prod-db \
--source-region us-east-1 \
--region us-west-2
# S3 cross-region replication
aws s3api put-bucket-replication \
--bucket source-bucket \
--replication-configuration file://replication.json
```
## DR Testing
## DR Strategies Comparison
```yaml
dr_test_schedule:
tabletop: Quarterly
component_failover: Monthly
full_failover: Annually
test_checklist:
- [ ] Verify backup integrity
- [ ] Test failover procedures
- [ ] Validate data consistency
- [ ] Measure actual RTO/RPO
- [ ] Document lessons learned
strategies:
backup_and_restore:
rto: "Hours"
rpo: "Hours (depends on backup frequency)"
cost: "$"
description: "Regular backups stored in DR region. Restore from backup when needed."
aws_services:
- "S3 cross-region replication for backups"
- "RDS automated snapshots copied to DR region"
- "AMI copies in DR region"
- "Terraform/CloudFormation for infrastructure rebuild"
pros: "Lowest cost, simplest to maintain"
cons: "Longest recovery time, highest data loss potential"
pilot_light:
rto: "Minutes to hours"
rpo: "Minutes"
cost: "$$"
description: "Core infrastructure running in DR region (databases replicated). Scale up compute on failover."
aws_services:
- "RDS cross-region read replica (always running)"
- "S3 cross-region replication"
- "AMIs pre-built in DR region"
- "Auto Scaling groups at zero/minimal capacity"
pros: "Fast database recovery, moderate cost"
cons: "Compute scale-up adds to recovery time"
warm_standby:
rto: "Minutes"
rpo: "Seconds to minutes"
cost: "$$$"
description: "Scaled-down but functional environment in DR region. Scale up on failover."
aws_services:
- "RDS cross-region read replica"
- "ECS/EKS running at reduced capacity"
- "Route53 health checks for automated DNS failover"
- "Global Accelerator for traffic management"
pros: "Fast failover, reduced risk"
cons: "Higher baseline cost for idle resources"
multi_site_active:
rto: "Near-zero"
rpo: "Near-zero"
cost: "$$$$"
description: "Active-active across regions. Traffic served from both regions simultaneously."
aws_services:
- "DynamoDB Global Tables or Aurora Global Database"
- "Route53 latency/weighted routing"
- "CloudFront with multi-origin"
- "Global Accelerator"
- "ECS/EKS in both regions"
pros: "Minimal downtime and data loss"
cons: "Highest cost, most complex to operate"
```
## AWS Cross-Region DR Implementation
```bash
# === Database Replication ===
# Create cross-region RDS read replica
aws rds create-db-instance-read-replica \
--db-instance-identifier prod-db-dr-replica \
--source-db-instance-identifier arn:aws:rds:us-east-1:123456789012:db:prod-db \
--db-instance-class db.r6g.large \
--region us-west-2 \
--kms-key-id arn:aws:kms:us-west-2:123456789012:alias/rds-dr-key \
--multi-az \
--tags Key=Purpose,Value=DR Key=Environment,Value=production
# Create Aurora Global Database for near-zero RPO
aws rds create-global-cluster \
--global-cluster-identifier prod-global-db \
--source-db-cluster-identifier arn:aws:rds:us-east-1:123456789012:cluster:prod-aurora-cluster \
--region us-east-1
# Add secondary region to Aurora Global Database
aws rds create-db-cluster \
--db-cluster-identifier prod-aurora-dr \
--global-cluster-identifier prod-global-db \
--engine aurora-postgresql \
--region us-west-2 \
--kms-key-id arn:aws:kms:us-west-2:123456789012:alias/aurora-dr-key
# === Storage Replication ===
# S3 cross-region replication
cat > /tmp/replication-config.json << 'EOF'
{
"Role": "arn:aws:iam::123456789012:role/s3-replication-role",
"Rules": [
{
"ID": "ReplicateAll",
"Status": "Enabled",
"Filter": {"Prefix": ""},
"Destination": {
"Bucket": "arn:aws:s3:::prod-data-dr-usw2",
"StorageClass": "STANDARD",
"EncryptionConfiguration": {
"ReplicaKmsKeyID": "arn:aws:kms:us-west-2:123456789012:alias/s3-dr-key"
}
},
"DeleteMarkerReplication": {"Status": "Enabled"}
}
]
}
EOF
aws s3api put-bucket-replication \
--bucket prod-data-use1 \
--replication-configuration file:///tmp/replication-config.json
# === DNS Failover ===
# Route53 health check for primary region
aws route53 create-health-check --caller-reference "prod-health-$(date +%s)" \
--health-check-config '{
"Type": "HTTPS",
"FullyQualifiedDomainName": "api.example.com",
"Port": 443,
"ResourcePath": "/health",
"RequestInterval": 10,
"FailureThreshold": 3,
"EnableSNI": true
}'
# Configure failover routing
aws route53 change-resource-record-sets --hosted-zone-id Z123456 \
--change-batch '{
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "api.example.com",
"Type": "A",
"SetIdentifier": "primary",
"Failover": "PRIMARY",
"AliasTarget": {
"HostedZoneId": "Z1234PRIMARY",
"DNSName": "primary-alb.us-east-1.elb.amazonaws.com",
"EvaluateTargetHealth": true
},
"HealthCheckId": "health-check-id-primary"
}
},
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "api.example.com",
"Type": "A",
"SetIdentifier": "secondary",
"Failover": "SECONDARY",
"AliasTarget": {
"HostedZoneId": "Z5678SECONDARY",
"DNSName": "dr-alb.us-west-2.elb.amazonaws.com",
"EvaluateTargetHealth": true
}
}
}
]
}'
```
## Failover Script
```bash
#!/usr/bin/env bash
# dr-failover.sh - Execute disaster recovery failover to DR region
set -euo pipefail
DR_REGION="us-west-2"
PRIMARY_REGION="us-east-1"
SLACK_WEBHOOK="${DR_SLACK_WEBHOOK}"
LOG_FILE="/var/log/dr-failover-$(date +%Y%m%d-%H%M%S).log"
log() {
echo "[$(date -u +%Y-%m-%dT%H:%M:%SZ)] $1" | tee -a "$LOG_FILE"
}
notify() {
curl -s -X POST "$SLACK_WEBHOOK" \
-H "Content-Type: application/json" \
-d "{\"text\":\"DR FAILOVER: $1\"}" > /dev/null
}
log "=== DR Failover Initiated ==="
notify "DR failover initiated to $DR_REGION"
# Step 1: Promote RDS read replica
log "Step 1: Promoting RDS read replica in $DR_REGION"
aws rds promote-read-replica \
--db-instance-identifier prod-db-dr-replica \
--region "$DR_REGION"
log "Waiting for RDS promotion to complete..."
aws rds wait db-instance-available \
--db-instance-identifier prod-db-dr-replica \
--region "$DR_REGION"
log "RDS promotion complete"
notify "RDS read replica promoted to primary in $DR_REGION"
# Step 2: Scale up application in DR region
log "Step 2: Scaling up application in $DR_REGION"
aws ecs update-service \
--cluster prod-cluster-dr \
--service api-service \
--desired-count 4 \
--region "$DR_REGION"
log "Waiting for ECS service to stabilize..."
aws ecs wait services-stable \
--cluster prod-cluster-dr \
--services api-service \
--region "$DR_REGION"
log "ECS service scaled up and stable"
notify "Application scaled up in $DR_REGION"
# Step 3: Verify health
log "Step 3: Verifying health in $DR_REGION"
for i in $(seq 1 10); do
STATUS=$(curl -s -o /dev/null -w "%{http_code}" "https://dr-alb.us-west-2.elb.amazonaws.com/health")
if [ "$STATUS" = "200" ]; then
log "Health check passed (attempt $i)"
break
fi
log "Health check failed (attempt $i, status $STATUS), retrying..."
sleep 10
done
if [ "$STATUS" != "200" ]; then
log "ERROR: Health check failed after 10 attempts"
notify "ALERT: DR health check failing - manual intervention required"
exit 1
fi
# Step 4: Update DNS (if not using automatic Route53 failover)
log "Step 4: DNS failover (Route53 automatic failover should handle this)"
log "Verifying DNS resolution..."
DR_IP=$(dig +short api.example.com)
log "api.example.com resolves to: $DR_IP"
# Step 5: Verify end-to-end
log "Step 5: End-to-end verification"
RESPONSE=$(curl -s "https://api.example.com/health")
log "Health response: $RESPONSE"
log "=== DR Failover Complete ==="
notify "DR failover to $DR_REGION complete. Service restored."
# Generate failover report
cat > "/var/log/dr-failover-report-$(date +%Y%m%d).md" << EOF
# DR Failover Report
- **Date:** $(date -u +%Y-%m-%dT%H:%M:%SZ)
- **Primary Region:** $PRIMARY_REGION
- **DR Region:** $DR_REGION
- **RTO Actual:** Calculate from incident declaration
- **RPO Actual:** Check replication lag at time of incident
- **Status:** Operational in DR region
- **Actions Required:**
- [ ] Monitor error rates and latency
- [ ] Plan failback when primary region is restored
- [ ] Conduct post-incident review
EOF
```
## DR Testing Procedures
```yaml
dr_test_types:
tabletop_exercise:
frequency: Quarterly
duration: "1-2 hours"
participants: "Engineering, SRE, management, communications"
process:
- Present a disaster scenario (region outage, data corruption, etc.)
- Walk through the response step by step
- Identify gaps in runbooks and communication plans
- Document action items
output: "Tabletop exercise report with findings and action items"
component_failover:
frequency: Monthly
duration: "1-4 hours"
scope: "Individual component failover (database, single service)"
process:
- Select component for testing
- Execute failover procedure from runbook
- Measure actual RTO and RPO
- Execute failback procedure
- Document results
output: "Component test report with measured RTO/RPO"
full_failover:
frequency: Annually
duration: "4-8 hours (scheduled maintenance window)"
scope: "Complete regional failover of all tier 1 and tier 2 services"
process:
1_preparation:
- Schedule maintenance window and notify stakeholders
- Verify DR environment is healthy
- Brief all participating teams
- Set up war room communication channel
2_execute:
- Simulate primary region failure
- Execute failover runbooks for all services
- Record timestamps at each milestone
3_verify:
- Run end-to-end test suite against DR environment
- Verify data consistency
- Check monitoring and alerting in DR region
- Confirm external integrations work
4_failback:
- Restore primary region
- Re-establish replication
- Execute failback to primary
- Verify data consistency post-failback
5_report:
- Document actual RTO and RPO for each service
- Compare against targets
- List all issues encountered
- Create action items for improvements
output: "Full DR test report with measured vs. target metrics"
dr_test_checklist:
before_test:
- [ ] Test plan documented and approved
- [ ] Maintenance window scheduled and communicated
- [ ] All DR runbooks reviewed and updated
- [ ] DR environment health verified
- [ ] Monitoring configured in DR region
- [ ] Communication channel established
- [ ] Rollback plan confirmed
during_test:
- [ ] Timestamps recorded for each step
- [ ] Screenshots captured for evidence
- [ ] Issues logged in real-time
- [ ] Data consistency verified
- [ ] External integrations tested
- [ ] Health checks passing in DR
after_test:
- [ ] Failback completed successfully
- [ ] Primary region replication re-established
- [ ] Data consistency verified post-failback
- [ ] Test report written with metrics
- [ ] Action items created and assigned
- [ ] Runbooks updated based on findings
- [ ] Results presented to management
```
## Terraform DR Infrastructure
```hcl
# DR region infrastructure
provider "aws" {
alias = "dr"
region = "us-west-2"
}
resource "aws_db_instance" "dr_replica" {
provider = aws.dr
identifier = "prod-db-dr-replica"
replicate_source_db = aws_db_instance.primary.arn
instance_class = "db.r6g.large"
storage_encrypted = true
kms_key_id = aws_kms_key.dr_rds.arn
multi_az = true
deletion_protection = true
skip_final_snapshot = false
tags = {
Purpose = "DR"
Environment = "production"
}
}
resource "aws_route53_health_check" "primary" {
fqdn = "primary-alb.us-east-1.elb.amazonaws.com"
port = 443
type = "HTTPS"
resource_path = "/health"
failure_threshold = 3
request_interval = 10
enable_sni = true
tags = {
Name = "primary-health-check"
}
}
resource "aws_route53_record" "failover_primary" {
zone_id = aws_route53_zone.main.zone_id
name = "api.example.com"
type = "A"
set_identifier = "primary"
failover_routing_policy {
type = "PRIMARY"
}
alias {
name = aws_lb.primary.dns_name
zone_id = aws_lb.primary.zone_id
evaluate_target_health = true
}
health_check_id = aws_route53_health_check.primary.id
}
resource "aws_route53_record" "failover_secondary" {
zone_id = aws_route53_zone.main.zone_id
name = "api.example.com"
type = "A"
set_identifier = "secondary"
failover_routing_policy {
type = "SECONDARY"
}
alias {
name = aws_lb.dr.dns_name
zone_id = aws_lb.dr.zone_id
evaluate_target_health = true
}
}
```
## DR Compliance Checklist
```yaml
dr_compliance_checklist:
planning:
- [ ] RTO and RPO targets defined per service tier
- [ ] DR strategy selected based on targets and budget
- [ ] DR architecture documented with diagrams
- [ ] Failover and failback runbooks written
- [ ] Communication plan for DR events documented
- [ ] DR roles and responsibilities assigned
implementation:
- [ ] Cross-region database replication configured
- [ ] Storage replication configured (S3, EBS snapshots)
- [ ] DNS failover routing configured
- [ ] DR region infrastructure provisioned (IaC)
- [ ] Monitoring and alerting configured in DR region
- [ ] Secrets and credentials available in DR region
testing:
- [ ] Tabletop exercises conducted quarterly
- [ ] Component failover tests conducted monthly
- [ ] Full failover test conducted annually
- [ ] Actual RTO/RPO measured and compared to targets
- [ ] Test results documented and reviewed
- [ ] Runbooks updated based on test findings
operational:
- [ ] Replication lag monitored with alerting
- [ ] DR environment health checked regularly
- [ ] Backup integrity verified monthly
- [ ] DR runbooks reviewed and updated quarterly
- [ ] DR test evidence archived for compliance audits
```
## Best Practices
- Regular DR testing
- Automate failover where possible
- Document all procedures
- Update runbooks after tests
- Define RTO and RPO targets based on business impact analysis, not technical convenience
- Choose the DR strategy that matches your targets and budget: do not over-engineer or under-invest
- Automate failover as much as possible to reduce human error and recovery time
- Test DR procedures regularly at increasing levels of complexity (tabletop, component, full)
- Measure actual RTO and RPO during tests and compare against targets every time
- Include failback procedures in your DR plan: getting back to normal is as important as failing over
- Monitor replication lag continuously and alert when it exceeds RPO thresholds
- Keep DR infrastructure managed by the same IaC as production to prevent configuration drift
- Practice DR in non-emergency conditions so the team is prepared when a real disaster occurs
- Archive DR test results as compliance evidence for SOC 2, HIPAA, and other frameworks
compliance/continuity/incident-management/SKILL.md
+426 -55
View File
@@ -9,82 +9,453 @@ metadata:
# Incident Management
Implement effective incident management processes.
Implement effective incident management processes including severity definitions, escalation matrices, war room procedures, and blameless post-mortem templates.
## Incident Severity
## When to Use
| Severity | Impact | Response | Example |
|----------|--------|----------|---------|
| SEV1 | Total outage | Immediate, all-hands | Site down |
| SEV2 | Major degradation | Urgent, on-call | Feature broken |
| SEV3 | Minor impact | Standard | Slow performance |
| SEV4 | Minimal | Next business day | Cosmetic issue |
- Establishing incident management processes for production systems
- Defining severity levels and escalation procedures
- Running war rooms and coordinating incident response
- Conducting blameless post-incident reviews
- Building on-call schedules and notification workflows
- Meeting compliance requirements for incident response (SOC 2, HIPAA, PCI DSS)
## Incident Process
## Severity Levels
```yaml
incident_workflow:
1_detect:
- Alerting triggers
- Customer reports
- Monitoring anomalies
2_triage:
- Severity assessment
- Impact determination
- Team notification
3_respond:
- Incident commander assigned
- Communication established
- Mitigation started
4_resolve:
- Root cause addressed
- Service restored
- Customer notified
5_review:
- Timeline documented
- Root cause analysis
- Action items created
severity_definitions:
SEV1_critical:
impact: "Complete service outage or data breach affecting all/most customers"
examples:
- Production site completely down
- Data breach confirmed or suspected
- Complete loss of a critical business function
- Security incident with active exploitation
response_time: "Immediate (within 5 minutes)"
update_frequency: "Every 15-30 minutes"
who_is_paged: "On-call engineer, engineering manager, incident commander, executive on-call"
communication: "Status page update, customer email, executive notification"
resolution_target: "< 1 hour to mitigate"
SEV2_major:
impact: "Major feature broken or severe degradation affecting many customers"
examples:
- Key feature completely non-functional
- Significant performance degradation (>5x latency)
- Data processing pipeline completely stalled
- Partial outage affecting a region or segment
response_time: "Within 15 minutes"
update_frequency: "Every 30-60 minutes"
who_is_paged: "On-call engineer, engineering manager"
communication: "Status page update if customer-facing"
resolution_target: "< 4 hours to mitigate"
SEV3_moderate:
impact: "Minor feature impaired or degradation affecting some customers"
examples:
- Non-critical feature broken
- Moderate performance degradation
- Elevated error rate (below threshold for SEV2)
- Single-customer impact on non-critical function
response_time: "Within 1 hour during business hours"
update_frequency: "Every 2-4 hours"
who_is_paged: "On-call engineer"
communication: "Internal only unless customer inquires"
resolution_target: "< 1 business day"
SEV4_low:
impact: "Cosmetic issue, minor inconvenience, or non-customer-facing problem"
examples:
- UI cosmetic bug
- Non-critical monitoring gap
- Internal tool degradation
- Documentation inaccuracy in production
response_time: "Next business day"
update_frequency: "As needed"
who_is_paged: "None (ticket created)"
communication: "None"
resolution_target: "Within sprint planning cycle"
```
## Incident Commander
## Escalation Matrix
```yaml
ic_responsibilities:
- Own incident resolution
- Coordinate response teams
- Manage communication
- Make escalation decisions
- Schedule post-mortem
escalation_matrix:
tier_1_on_call_engineer:
reached_via: "PagerDuty / OpsGenie alert"
responsibilities:
- Acknowledge alert within 5 minutes
- Assess severity and impact
- Begin troubleshooting
- Escalate to Tier 2 if unable to resolve within 30 minutes (SEV1/2)
escalation_trigger: "Cannot resolve, needs additional expertise, or severity upgrade"
tier_2_team_lead_or_sme:
reached_via: "PagerDuty escalation or direct page"
responsibilities:
- Provide subject matter expertise
- Assist with diagnosis and resolution
- Coordinate with other teams if cross-service issue
- Escalate to Tier 3 if broader coordination needed
escalation_trigger: "Multi-service issue, needs executive decision, or customer-facing SEV1"
tier_3_engineering_management:
reached_via: "PagerDuty escalation or direct call"
responsibilities:
- Assign incident commander (if not already)
- Allocate additional resources
- Make business decisions (feature disable, rollback, etc.)
- Coordinate external communication
escalation_trigger: "Business impact decision, extended outage, or PR/legal concern"
tier_4_executive:
reached_via: "Direct phone call"
responsibilities:
- Authorize extraordinary measures
- Manage board/investor communication
- Approve public statements
- Engage external resources (vendors, consultants)
escalation_trigger: "Major breach, extended SEV1, regulatory or legal implication"
time_based_escalation:
sev1:
"15 min no ack": "Re-page on-call + backup on-call"
"30 min unresolved": "Page team lead"
"1 hour unresolved": "Page engineering manager + executive on-call"
"2 hours unresolved": "All-hands engineering involvement"
sev2:
"30 min no ack": "Re-page on-call + backup on-call"
"1 hour unresolved": "Page team lead"
"4 hours unresolved": "Page engineering manager"
```
## Post-Incident Review
## War Room Procedures
```yaml
war_room:
activation: "Automatically for SEV1, on-demand for SEV2"
setup:
communication_channel:
primary: "Dedicated Slack channel (#incident-YYYY-MM-DD-brief-name)"
voice: "Zoom/Google Meet bridge (persistent link)"
backup: "Phone conference bridge"
channel_rules:
- "Only incident-related communication in the channel"
- "Use threads for side discussions"
- "Prefix messages with role (IC:, COMMS:, ENG:)"
roles:
incident_commander:
responsibilities:
- Own the incident from declaration to resolution
- Coordinate all response activities
- Make decisions on response actions
- Assign tasks to responders
- Determine when incident is resolved
- Schedule post-mortem
selection: "On-call IC roster, or senior engineer who declares the incident"
communications_lead:
responsibilities:
- Draft and publish status page updates
- Coordinate customer notifications
- Handle internal stakeholder updates
- Manage executive communication
- Document timeline in real-time
selection: "Designated from on-call comms roster or engineering manager"
technical_lead:
responsibilities:
- Lead technical diagnosis and troubleshooting
- Coordinate technical responders
- Recommend mitigation and resolution actions
- Verify fix effectiveness
selection: "Senior engineer with relevant system expertise"
scribe:
responsibilities:
- Document all actions, decisions, and findings
- Maintain real-time timeline
- Record who did what and when
- Capture screenshots and log excerpts
selection: "Any available team member (can be rotated)"
workflow:
1_declare:
- "IC declares incident with severity level"
- "War room channel and bridge created"
- "Roles assigned"
- "First status update posted"
2_assess:
- "Determine scope and customer impact"
- "Identify affected systems and services"
- "Establish working hypothesis"
3_mitigate:
- "Focus on restoring service first, root cause second"
- "IC approves all changes to production"
- "Changes documented in real-time"
- "Rollback if mitigation makes things worse"
4_resolve:
- "Confirm service restored to normal"
- "Verify monitoring shows healthy metrics"
- "IC declares incident resolved"
- "Final status page update"
5_follow_up:
- "Schedule post-mortem within 48 hours"
- "Assign action items from immediate findings"
- "Send internal summary"
```
## On-Call Configuration
```yaml
on_call_schedule:
rotation_structure:
primary:
rotation: "Weekly"
handoff: "Monday 10:00 AM local time"
team_size: "Minimum 5 engineers in rotation"
secondary:
rotation: "Weekly (offset from primary)"
activation: "If primary does not acknowledge within 10 minutes"
expectations:
response_time: "Acknowledge alert within 5 minutes"
availability: "Reachable by phone and laptop within 15 minutes"
handoff: "Document any ongoing issues during handoff"
compensation: "Per company on-call compensation policy"
health:
max_consecutive_weeks: 2
minimum_gap_between_rotations: "2 weeks"
post_incident_rest: "If engaged for 4+ hours overnight, late start next day"
burnout_monitoring: "Track pages per person per week, rebalance if needed"
pagerduty_configuration:
escalation_policy:
- level_1:
target: "Primary on-call"
timeout: "5 minutes"
- level_2:
target: "Secondary on-call"
timeout: "10 minutes"
- level_3:
target: "Engineering manager"
timeout: "15 minutes"
notification_rules:
high_urgency:
- "Push notification immediately"
- "Phone call after 1 minute"
- "SMS after 2 minutes"
low_urgency:
- "Push notification"
- "Email after 5 minutes"
```
## Post-Mortem Template
```markdown
## Incident Summary
- Duration:
- Impact:
- Severity:
# Post-Incident Review: [Incident Title]
## Timeline
**Date:** YYYY-MM-DD
**Severity:** SEV[1-4]
**Duration:** [Start time] to [End time] ([X hours Y minutes])
**Incident Commander:** [Name]
**Author:** [Name]
**Status:** Draft / In Review / Final
## Executive Summary
[2-3 sentence summary of what happened, the impact, and the resolution]
## Impact
- **Customer impact:** [Number/percentage of customers affected, what they experienced]
- **Duration of impact:** [How long customers were affected]
- **Revenue impact:** [Estimated financial impact, if applicable]
- **Data impact:** [Any data loss or corruption]
- **SLA impact:** [Any SLA breaches]
## Timeline (all times UTC)
| Time | Event |
|------|-------|
| HH:MM | [First anomaly detected by monitoring] |
| HH:MM | [Alert fired / customer report received] |
| HH:MM | [On-call engineer acknowledged] |
| HH:MM | [Incident declared at SEV level] |
| HH:MM | [War room established] |
| HH:MM | [Root cause identified] |
| HH:MM | [Mitigation applied] |
| HH:MM | [Service restored] |
| HH:MM | [Incident resolved] |
## Root Cause
[Detailed technical explanation of what caused the incident]
## What Went Well
## Detection
- **How was the incident detected?** [Monitoring alert / customer report / manual observation]
- **Time to detect:** [Time from first anomaly to detection]
- **Could we have detected sooner?** [Yes/No, with explanation]
## What Could Be Improved
## Response
- **What went well:**
- [List things that worked effectively during response]
- [E.g., "Runbook for database failover was accurate and followed successfully"]
- [E.g., "Communication to customers was timely and clear"]
- **What could be improved:**
- [List things that slowed or hindered response]
- [E.g., "Took 20 minutes to identify the correct service owner"]
- [E.g., "Monitoring did not alert on the specific failure mode"]
## Contributing Factors
[List all factors that contributed to the incident occurring or being worse than it could have been. This is not about blame - it is about understanding the system.]
1. [Factor 1: e.g., "Configuration change was not tested in staging"]
2. [Factor 2: e.g., "Alert threshold was too high to catch gradual degradation"]
3. [Factor 3: e.g., "No circuit breaker between Service A and Service B"]
## Action Items
| Item | Owner | Due Date |
| ID | Action | Owner | Priority | Due Date | Status |
|----|--------|-------|----------|----------|--------|
| 1 | [Preventive action] | [Name] | P1 | YYYY-MM-DD | Open |
| 2 | [Detection improvement] | [Name] | P2 | YYYY-MM-DD | Open |
| 3 | [Process improvement] | [Name] | P2 | YYYY-MM-DD | Open |
| 4 | [Runbook update] | [Name] | P3 | YYYY-MM-DD | Open |
## Lessons Learned
[Key takeaways that should be shared broadly]
## Appendix
- [Link to monitoring dashboards during incident]
- [Link to relevant log queries]
- [Link to war room channel archive]
```
## Post-Mortem Process
```yaml
post_mortem_process:
scheduling:
sev1: "Within 48 hours of resolution"
sev2: "Within 1 week of resolution"
sev3: "Within 2 weeks (optional, based on learning potential)"
sev4: "Not required"
meeting_format:
duration: "60-90 minutes"
attendees:
required: "IC, technical lead, scribe, involved engineers"
optional: "Engineering manager, product manager, affected team leads"
agenda:
- "5 min: Review timeline and facts"
- "15 min: Walk through root cause and contributing factors"
- "15 min: Discuss what went well"
- "15 min: Discuss what could be improved"
- "15 min: Define and assign action items"
- "5 min: Identify lessons learned and sharing plan"
principles:
- "Blameless: Focus on systems and processes, not individuals"
- "Factual: Base discussion on data, logs, and observations"
- "Forward-looking: Prioritize preventive actions over assigning fault"
- "Complete: Address detection, response, and prevention"
- "Actionable: Every finding should produce a tracked action item"
action_item_tracking:
- "All action items entered into issue tracker (Jira, GitHub Issues)"
- "Priority assigned based on risk reduction potential"
- "Owner assigned with due date"
- "Reviewed in team standups and sprint planning"
- "Tracked to completion"
- "Monthly review of open post-mortem action items"
```
## Incident Metrics
```yaml
incident_metrics:
mttr:
name: "Mean Time to Resolve"
definition: "Average time from incident detection to resolution"
target: "SEV1: <1h, SEV2: <4h"
trending: "Track monthly, aim for improvement"
mttd:
name: "Mean Time to Detect"
definition: "Average time from incident start to detection"
target: "< 5 minutes for SEV1/2"
trending: "Monitors effectiveness of alerting"
mtta:
name: "Mean Time to Acknowledge"
definition: "Average time from alert to engineer acknowledgment"
target: "< 5 minutes"
trending: "Monitors on-call responsiveness"
incident_frequency:
name: "Incidents per week/month by severity"
target: "Trending downward"
trending: "Monitors system reliability improvement"
action_item_completion:
name: "Post-mortem action item completion rate"
target: "> 90% completed on time"
trending: "Monitors follow-through on improvements"
recurring_incidents:
name: "Percentage of incidents with same root cause as previous incident"
target: "< 10%"
trending: "Monitors effectiveness of preventive actions"
```
## Incident Management Checklist
```yaml
incident_management_checklist:
process_setup:
- [ ] Severity levels defined with clear criteria
- [ ] Escalation matrix documented
- [ ] On-call schedule established and staffed
- [ ] War room procedures documented
- [ ] Post-mortem template created
- [ ] Communication templates prepared (status page, email)
- [ ] Incident management tool configured (PagerDuty, OpsGenie)
per_incident:
- [ ] Incident declared with severity level
- [ ] War room established (SEV1/2)
- [ ] Roles assigned (IC, comms, technical lead, scribe)
- [ ] Timeline maintained in real-time
- [ ] Status page updated (customer-facing impact)
- [ ] Stakeholders notified per communication plan
- [ ] Resolution verified with monitoring
- [ ] Post-mortem scheduled
- [ ] Post-mortem conducted and published
- [ ] Action items tracked to completion
compliance:
- [ ] All SEV1/2 incidents have post-mortems
- [ ] Incident log maintained for audit evidence
- [ ] Metrics reported monthly
- [ ] On-call health monitored (pages per person)
- [ ] Annual incident response training conducted
- [ ] Annual incident response plan test completed
```
## Best Practices
- Clear severity definitions
- Defined escalation paths
- Blameless post-mortems
- Action item tracking
- Regular training
- Define severity levels with concrete examples so there is no ambiguity during an active incident
- Implement time-based escalation: if the on-call does not acknowledge, automatically escalate
- Focus on mitigation first, root cause second: restore service before investigating why it failed
- Run blameless post-mortems: the goal is to improve systems, not to assign fault to individuals
- Track post-mortem action items to completion: an unfinished action item means the same incident can recur
- Monitor incident metrics (MTTR, MTTD, frequency) as leading indicators of system reliability
- Protect on-call health: track page volume per person and redistribute if someone is overburdened
- Separate the incident commander role from the technical lead role in SEV1/2 incidents
- Practice incident response regularly with game days or chaos engineering exercises
- Archive incident records and post-mortems for compliance evidence and organizational learning
compliance/continuity/runbook-creation/SKILL.md
+441 -52
View File
@@ -9,88 +9,477 @@ metadata:
# Runbook Creation
Create effective operational runbooks and procedures.
Create effective operational runbooks, standard operating procedures, and
troubleshooting guides that any on-call engineer can follow under pressure.
## Runbook Structure
## Runbook Template — Full Structure
```markdown
# Runbook: [Service/Process Name]
````markdown
# Runbook: [Service / Process Name]
**Owner:** [Team or individual]
**Last Reviewed:** YYYY-MM-DD
**Version:** X.Y
**Severity if unavailable:** SEV[1-4]
---
## Overview
Brief description of the service and runbook purpose.
Brief description of the service, why this runbook exists, and when to
use it.
## Prerequisites
- Required access
- Tools needed
- Knowledge required
- [ ] Required access / IAM role: [details]
- [ ] Tools installed: [kubectl, aws-cli, psql, etc.]
- [ ] VPN connected to [environment]
- [ ] Communication channel open: [Slack #channel]
## Procedure
Step-by-step instructions with commands.
## Verification
How to confirm success.
### Step 1 — [Action Name]
## Rollback
Steps to undo if needed.
[Explanation of what this step does and why.]
## Escalation
When and how to escalate.
## Related Runbooks
Links to related procedures.
```bash
# command here
```
## Example Runbook
**Expected output:** [describe what success looks like]
```markdown
# Runbook: Database Failover
### Step 2 — [Action Name]
```bash
# command here
```
**Expected output:** [description]
*(Continue with numbered steps...)*
## Verification
How to confirm the procedure succeeded:
- [ ] [Check 1 — e.g., health endpoint returns 200]
- [ ] [Check 2 — e.g., no errors in logs for 5 minutes]
- [ ] [Check 3 — e.g., metrics return to baseline]
## Rollback
If the procedure fails or causes unexpected issues:
### Rollback Step 1
```bash
# rollback command
```
### Rollback Step 2
```bash
# rollback command
```
## Troubleshooting
| Symptom | Likely Cause | Resolution |
|---------|-------------|------------|
| [symptom 1] | [cause] | [fix] |
| [symptom 2] | [cause] | [fix] |
## Escalation
If unresolved after [X] minutes:
- **Primary:** @[team-lead] — [phone/Slack]
- **Secondary:** @[manager] — [phone/Slack]
## Related Runbooks
- [Link to related runbook 1]
- [Link to related runbook 2]
## Change Log
| Date | Author | Change |
|------|--------|--------|
| YYYY-MM-DD | [Name] | Initial version |
````
## Example Runbook — Database Failover
````markdown
# Runbook: PostgreSQL Database Failover
**Owner:** Platform / DBA team
**Last Reviewed:** 2025-06-15
**Version:** 2.1
**Severity if unavailable:** SEV1
---
## Overview
Procedure to failover PostgreSQL to replica.
Failover the primary PostgreSQL instance to the synchronous replica when
the primary is unreachable or degraded. This runbook covers both planned
(maintenance) and unplanned (emergency) failover.
## Prerequisites
- [ ] DBA access to primary and replica
- [ ] VPN connected
- [ ] DBA or SRE-level access to primary and replica hosts
- [ ] `psql` client installed (v14+)
- [ ] VPN connected to production network
- [ ] Slack channel #db-ops open
- [ ] Confirm replica is in sync: replication lag < 1 MB
## Procedure
### 1. Verify Replica Status
\`\`\`bash
psql -h replica -c "SELECT pg_is_in_recovery();"
# Should return 't'
\`\`\`
### Step 1 — Verify Replica Health
### 2. Stop Application Writes
\`\`\`bash
kubectl scale deployment app --replicas=0
\`\`\`
```bash
psql -h replica.db.internal -U dba -d postgres -c \
"SELECT pg_is_in_recovery(), pg_last_wal_replay_lsn();"
```
### 3. Promote Replica
\`\`\`bash
psql -h replica -c "SELECT pg_promote();"
\`\`\`
**Expected output:** `pg_is_in_recovery = t`, LSN advancing.
### 4. Update DNS
\`\`\`bash
aws route53 change-resource-record-sets ...
\`\`\`
### Step 2 — Stop Application Writes
```bash
kubectl scale deployment api-server --replicas=0 -n production
kubectl scale deployment worker --replicas=0 -n production
```
**Expected output:** Deployments scaled to 0 pods.
### Step 3 — Confirm Write Quiesce
```bash
psql -h primary.db.internal -U dba -d postgres -c \
"SELECT count(*) FROM pg_stat_activity WHERE state = 'active' AND query !~ 'pg_stat';"
```
**Expected output:** Count = 0 (no active queries).
### Step 4 — Promote Replica
```bash
psql -h replica.db.internal -U dba -d postgres -c "SELECT pg_promote();"
```
Wait up to 30 seconds, then confirm:
```bash
psql -h replica.db.internal -U dba -d postgres -c "SELECT pg_is_in_recovery();"
```
**Expected output:** `pg_is_in_recovery = f` (no longer a replica).
### Step 5 — Update DNS
```bash
aws route53 change-resource-record-sets \
--hosted-zone-id Z1234567890 \
--change-batch '{
"Changes": [{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "db.internal.example.com",
"Type": "CNAME",
"TTL": 60,
"ResourceRecords": [{"Value": "replica.db.internal"}]
}
}]
}'
```
### Step 6 — Restart Application
```bash
kubectl scale deployment api-server --replicas=6 -n production
kubectl scale deployment worker --replicas=4 -n production
```
## Verification
- [ ] Application connects to new primary
- [ ] No replication lag errors
- [ ] Transactions completing
- [ ] `psql -h db.internal.example.com -c "SELECT 1;"` returns successfully
- [ ] Application logs show successful DB connections (no errors for 5 min)
- [ ] Transaction throughput returns to baseline on Grafana dashboard
- [ ] No replication-lag alerts firing
## Rollback
If the promoted replica has issues, restore from the most recent backup:
```bash
# Restore latest automated snapshot (RDS example)
aws rds restore-db-instance-from-db-snapshot \
--db-instance-identifier prod-db-restored \
--db-snapshot-identifier prod-db-latest-snapshot
```
## Escalation
If issues persist after 15 minutes, escalate to:
- Primary: @dba-lead
- Secondary: @platform-oncall
If unresolved after 15 minutes:
- **Primary:** @dba-lead — +1-555-0101
- **Secondary:** @platform-oncall — +1-555-0102
````
## Automation Scripts for Common Operations
### Service Health Check
```bash
#!/usr/bin/env bash
# health-check.sh — Check health of critical services
set -euo pipefail
SERVICES=(
"https://api.example.com/healthz"
"https://app.example.com/healthz"
"https://admin.example.com/healthz"
)
EXIT_CODE=0
for url in "${SERVICES[@]}"; do
HTTP_CODE=$(curl -so /dev/null -w '%{http_code}' --max-time 5 "$url" 2>/dev/null || echo "000")
if [ "$HTTP_CODE" -eq 200 ]; then
printf " OK %s\n" "$url"
else
printf " FAIL %s (HTTP %s)\n" "$url" "$HTTP_CODE"
EXIT_CODE=1
fi
done
exit $EXIT_CODE
```
### Log Collection for Incident Investigation
```bash
#!/usr/bin/env bash
# collect-logs.sh — Gather logs from multiple sources for incident review
set -euo pipefail
INCIDENT_ID="${1:?Usage: collect-logs.sh <incident-id>}"
OUTDIR="/tmp/incident-${INCIDENT_ID}"
mkdir -p "$OUTDIR"
echo "Collecting logs for incident $INCIDENT_ID..."
# Kubernetes pod logs (last 30 min)
kubectl logs -l app=api-server -n production --since=30m \
> "${OUTDIR}/api-server-pods.log" 2>&1
# CloudWatch Logs (last 30 min)
aws logs filter-log-events \
--log-group-name /ecs/production/api \
--start-time "$(date -d '30 minutes ago' +%s)000" \
--output text > "${OUTDIR}/cloudwatch-api.log" 2>&1
# Database slow query log
psql -h db.internal -U dba -d postgres -c \
"SELECT * FROM pg_stat_activity WHERE state != 'idle' ORDER BY query_start;" \
> "${OUTDIR}/db-active-queries.log" 2>&1
# System resource snapshot
kubectl top pods -n production > "${OUTDIR}/pod-resources.log" 2>&1
echo "Logs saved to $OUTDIR"
tar czf "${OUTDIR}.tar.gz" -C /tmp "incident-${INCIDENT_ID}"
echo "Archive: ${OUTDIR}.tar.gz"
```
### Certificate Expiry Check
```bash
#!/usr/bin/env bash
# cert-check.sh — Warn if TLS certificates expire within 30 days
set -euo pipefail
DOMAINS=(
"api.example.com"
"app.example.com"
"admin.example.com"
)
WARN_DAYS=30
TODAY=$(date +%s)
EXIT_CODE=0
for domain in "${DOMAINS[@]}"; do
EXPIRY=$(echo | openssl s_client -servername "$domain" -connect "${domain}:443" 2>/dev/null \
| openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2)
EXPIRY_EPOCH=$(date -d "$EXPIRY" +%s 2>/dev/null || echo 0)
DAYS_LEFT=$(( (EXPIRY_EPOCH - TODAY) / 86400 ))
if [ "$DAYS_LEFT" -lt "$WARN_DAYS" ]; then
printf " WARN %s expires in %d days (%s)\n" "$domain" "$DAYS_LEFT" "$EXPIRY"
EXIT_CODE=1
else
printf " OK %s — %d days remaining\n" "$domain" "$DAYS_LEFT"
fi
done
exit $EXIT_CODE
```
### Disk Space Cleanup
```bash
#!/usr/bin/env bash
# disk-cleanup.sh — Free disk space on a host
set -euo pipefail
echo "=== Disk Usage Before ==="
df -h /
# Remove old journal logs (> 7 days)
journalctl --vacuum-time=7d 2>/dev/null || true
# Clean Docker artifacts
docker system prune -f --volumes 2>/dev/null || true
# Remove old log files
find /var/log -name "*.gz" -mtime +7 -delete 2>/dev/null || true
find /tmp -type f -mtime +3 -delete 2>/dev/null || true
echo "=== Disk Usage After ==="
df -h /
```
## Runbook Review Checklist
Use this checklist every time a runbook is created or updated.
```yaml
content_review:
- [ ] Title clearly identifies the service and operation
- [ ] Overview explains WHEN and WHY to use this runbook
- [ ] Prerequisites list all required access, tools, and setup
- [ ] Every step has a concrete command (no vague instructions)
- [ ] Expected output is documented for each step
- [ ] Verification section confirms success with specific checks
- [ ] Rollback section exists and has been tested
- [ ] Escalation contacts are current (names, phones, Slack handles)
- [ ] Troubleshooting table covers the top 3-5 known failure modes
usability_review:
- [ ] A new team member can follow the runbook without tribal knowledge
- [ ] Steps are numbered and sequential (no branching without clear labels)
- [ ] Commands can be copy-pasted (no placeholder values without explanation)
- [ ] Time estimates included for long-running steps
- [ ] No jargon or acronyms used without definition
maintenance_review:
- [ ] Owner and last-reviewed date are set
- [ ] Version number incremented
- [ ] Change log entry added
- [ ] Related runbooks section is up to date
- [ ] Links to dashboards and docs are valid (not broken)
```
## Runbook Testing Procedures
```yaml
testing_strategy:
dry_run:
frequency: "Every time a runbook is created or substantially edited"
method: "Walk through each step in a staging environment"
goal: "Verify commands work and output matches documentation"
peer_review:
frequency: "Every edit"
method: "Another engineer follows the runbook in staging without help"
goal: "Confirm the runbook is self-contained and unambiguous"
scheduled_validation:
frequency: "Quarterly"
method: "SRE team picks 5 runbooks at random, executes in staging"
goal: "Catch runbooks that have drifted from production reality"
incident_triggered:
trigger: "Any time a runbook is used in a real incident"
method: "Post-mortem includes runbook accuracy assessment"
goal: "Capture improvements while the experience is fresh"
automation_testing:
method: "CI pipeline validates bash scripts with shellcheck and dry-run"
example: |
# .github/workflows/runbook-lint.yml
name: Lint Runbook Scripts
on: [pull_request]
jobs:
shellcheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: ShellCheck
run: |
find runbooks/ -name "*.sh" -exec shellcheck {} +
```
## Versioning Strategy
```yaml
versioning:
storage: "Git repository — one directory per service, one file per runbook"
naming: "runbooks/<service>/<operation>.md"
branching: "PRs required for all changes; reviewed by service owner"
version_scheme:
format: "MAJOR.MINOR"
major_bump: "Procedure changes that alter the steps or their order"
minor_bump: "Clarifications, typo fixes, updated contact info"
directory_layout: |
runbooks/
api-server/
deploy.md
rollback.md
scale-up.md
database/
failover.md
backup-restore.md
vacuum-maintenance.md
infrastructure/
dns-update.md
certificate-renewal.md
disk-cleanup.md
review_requirements:
- PR must be approved by the service owner
- CI must pass (shellcheck for scripts, markdown lint)
- Reviewer confirms they can follow the steps independently
retention: "Git history serves as full audit trail — never delete old versions"
```
## Runbook Index Template
Keep a top-level index so engineers can find the right runbook quickly.
```markdown
# Runbook Index
| Service | Runbook | Severity | Owner | Last Tested |
|---------|---------|----------|-------|-------------|
| API Server | [Deploy](api-server/deploy.md) | — | @platform | 2025-05-01 |
| API Server | [Rollback](api-server/rollback.md) | SEV1 | @platform | 2025-05-01 |
| Database | [Failover](database/failover.md) | SEV1 | @dba | 2025-04-15 |
| Database | [Backup Restore](database/backup-restore.md) | SEV2 | @dba | 2025-04-15 |
| Infra | [DNS Update](infrastructure/dns-update.md) | SEV2 | @sre | 2025-06-01 |
| Infra | [Cert Renewal](infrastructure/certificate-renewal.md) | SEV3 | @sre | 2025-06-01 |
```
## Best Practices
- Keep procedures simple and clear
- Include verification steps
- Test runbooks regularly
- Version control runbooks
- Include troubleshooting tips
- Write runbooks for the engineer at 3 AM — clear, sequential, copy-pasteable
- Include expected output so the operator knows if a step succeeded
- Always provide a rollback path; every action should be reversible
- Test runbooks in staging before they are needed in production
- Keep runbooks in version control alongside the code they support
- Assign an owner to every runbook; ownerless runbooks rot fast
- After every incident, update the relevant runbook with lessons learned
- Automate repetitive runbook steps into scripts, but keep the runbook as
the orchestration guide so operators understand the "why"
compliance/frameworks/fedramp-compliance/SKILL.md
+398 -40
View File
@@ -9,63 +9,421 @@ metadata:
# FedRAMP Compliance
Implement FedRAMP requirements for federal cloud services.
Implement FedRAMP (Federal Risk and Authorization Management Program) requirements for cloud service providers serving US federal agencies.
## When to Use
- Pursuing FedRAMP authorization for a cloud service offering
- Implementing NIST 800-53 security controls for federal workloads
- Establishing continuous monitoring (ConMon) processes
- Managing Plan of Action and Milestones (POA&M) tracking
- Preparing for a Third-Party Assessment Organization (3PAO) audit
- Operating a FedRAMP-authorized system and maintaining authorization
## Impact Levels
```yaml
levels:
impact_levels:
low:
controls: ~125
use_case: Public data
control_count: ~125
use_case: "Publicly available federal information"
examples:
- Public-facing websites with no sensitive data
- Open data portals
- Marketing and informational systems
data_types: "No PII, no CUI, publicly releasable only"
authorization_path: "FedRAMP Tailored (Li-SaaS) or standard Low"
moderate:
controls: ~325
use_case: CUI, most federal systems
control_count: ~325
use_case: "Most federal systems, including CUI"
examples:
- Email and collaboration platforms
- Case management systems
- Financial management systems
- HR and personnel systems
data_types: "CUI, PII, law enforcement sensitive (LES)"
authorization_path: "Agency or JAB P-ATO"
note: "~80% of FedRAMP authorizations are at Moderate"
high:
controls: ~425
use_case: Law enforcement, emergency services
control_count: ~425
use_case: "High-impact federal systems"
examples:
- Law enforcement and criminal justice systems
- Emergency services and public safety
- Financial systems with significant impact
- Healthcare systems with PHI
data_types: "Classified-adjacent, life-safety, critical infrastructure"
authorization_path: "JAB P-ATO required"
```
## NIST 800-53 Families
## NIST 800-53 Control Families
```yaml
control_families:
AC: Access Control
AU: Audit and Accountability
AT: Awareness and Training
CM: Configuration Management
CP: Contingency Planning
IA: Identification and Authentication
IR: Incident Response
MA: Maintenance
MP: Media Protection
PE: Physical Protection
PL: Planning
PS: Personnel Security
RA: Risk Assessment
CA: Assessment and Authorization
SC: System and Communications Protection
SI: System and Information Integrity
SA: System and Services Acquisition
PM: Program Management
AC:
name: "Access Control"
key_controls:
AC-2: "Account Management - manage system accounts lifecycle"
AC-3: "Access Enforcement - enforce approved authorizations"
AC-6: "Least Privilege - employ principle of least privilege"
AC-17: "Remote Access - establish usage restrictions for remote access"
implementation_notes: "Map to IAM policies, RBAC, MFA enforcement"
AU:
name: "Audit and Accountability"
key_controls:
AU-2: "Audit Events - define auditable events"
AU-3: "Content of Audit Records - ensure records contain required info"
AU-6: "Audit Review, Analysis, and Reporting"
AU-12: "Audit Generation - generate audit records"
implementation_notes: "Map to CloudTrail, CloudWatch Logs, SIEM"
AT:
name: "Awareness and Training"
key_controls:
AT-2: "Security Awareness Training - provide training to users"
AT-3: "Role-Based Security Training - for personnel with security roles"
implementation_notes: "Annual security training, role-specific training"
CM:
name: "Configuration Management"
key_controls:
CM-2: "Baseline Configuration - develop and maintain baseline"
CM-6: "Configuration Settings - establish mandatory settings"
CM-7: "Least Functionality - restrict to essential capabilities"
CM-8: "Information System Component Inventory"
implementation_notes: "Map to AWS Config, SSM, hardened AMIs"
CP:
name: "Contingency Planning"
key_controls:
CP-2: "Contingency Plan - develop and maintain plan"
CP-4: "Contingency Plan Testing - test plan annually"
CP-9: "Information System Backup"
CP-10: "Information System Recovery and Reconstitution"
implementation_notes: "Map to DR plan, backup strategy, failover testing"
IA:
name: "Identification and Authentication"
key_controls:
IA-2: "Identification and Authentication (Org Users)"
IA-5: "Authenticator Management"
IA-8: "Identification and Authentication (Non-Org Users)"
implementation_notes: "Map to SSO, MFA, certificate-based auth, PIV/CAC"
IR:
name: "Incident Response"
key_controls:
IR-2: "Incident Response Training"
IR-4: "Incident Handling - implement incident handling capability"
IR-6: "Incident Reporting - report incidents to US-CERT"
IR-8: "Incident Response Plan"
implementation_notes: "US-CERT reporting within 1 hour for federal incidents"
MA:
name: "Maintenance"
key_controls:
MA-2: "Controlled Maintenance"
MA-4: "Nonlocal Maintenance - authorize nonlocal maintenance"
implementation_notes: "Patching procedures, remote maintenance controls"
MP:
name: "Media Protection"
key_controls:
MP-2: "Media Access - restrict access to media"
MP-6: "Media Sanitization - sanitize media prior to disposal"
implementation_notes: "Encryption at rest, secure disposal procedures"
PE:
name: "Physical and Environmental Protection"
key_controls:
PE-2: "Physical Access Authorizations"
PE-3: "Physical Access Control"
PE-6: "Monitoring Physical Access"
implementation_notes: "Inherit from CSP for IaaS/PaaS, document inheritance"
PL:
name: "Planning"
key_controls:
PL-2: "System Security Plan (SSP)"
implementation_notes: "SSP is the core FedRAMP deliverable"
PS:
name: "Personnel Security"
key_controls:
PS-3: "Personnel Screening"
PS-4: "Personnel Termination"
PS-5: "Personnel Transfer"
implementation_notes: "Background checks, access revocation on termination"
RA:
name: "Risk Assessment"
key_controls:
RA-3: "Risk Assessment - conduct risk assessment"
RA-5: "Vulnerability Scanning"
implementation_notes: "Annual risk assessment, monthly vulnerability scans"
CA:
name: "Security Assessment and Authorization"
key_controls:
CA-2: "Security Assessments"
CA-6: "Security Authorization"
CA-7: "Continuous Monitoring"
implementation_notes: "Annual assessment by 3PAO, ConMon program"
SC:
name: "System and Communications Protection"
key_controls:
SC-7: "Boundary Protection"
SC-8: "Transmission Confidentiality and Integrity"
SC-12: "Cryptographic Key Establishment and Management"
SC-13: "Cryptographic Protection - FIPS 140-2 validated"
SC-28: "Protection of Information at Rest"
implementation_notes: "FIPS 140-2 validated modules required"
SI:
name: "System and Information Integrity"
key_controls:
SI-2: "Flaw Remediation"
SI-3: "Malicious Code Protection"
SI-4: "Information System Monitoring"
SI-5: "Security Alerts, Advisories, and Directives"
implementation_notes: "Patching SLAs, antimalware, IDS/IPS, SIEM"
SA:
name: "System and Services Acquisition"
key_controls:
SA-4: "Acquisition Process - security requirements in contracts"
SA-9: "External Information System Services"
SA-11: "Developer Security Testing"
implementation_notes: "Supply chain risk management, SBOM"
PM:
name: "Program Management"
key_controls:
PM-1: "Information Security Program Plan"
PM-9: "Risk Management Strategy"
implementation_notes: "Organization-wide security program"
```
## Continuous Monitoring
## System Security Plan (SSP) Outline
```yaml
conmon:
vulnerability_scans: Monthly
penetration_tests: Annual
poa_m_updates: Monthly
security_assessment: Annual
ssp_sections:
section_1: "Information System Name and Title"
section_2: "Information System Categorization (FIPS 199)"
section_3: "Information System Owner"
section_4: "Authorizing Official"
section_5: "Other Designated Contacts"
section_6: "Assignment of Security Responsibility"
section_7: "Information System Operational Status"
section_8: "Information System Type (cloud service model)"
section_9: "General System Description"
section_10: "System Environment and Special Considerations"
section_11: "System Interconnections"
section_12: "Laws, Regulations, Policies Applicable"
section_13: "Minimum Security Controls"
key_attachments:
- "Control Implementation Summary (CIS) workbook"
- "Network architecture diagrams"
- "Data flow diagrams"
- "Interconnection security agreements (ISAs)"
- "Incident response plan"
- "Contingency plan"
- "Configuration management plan"
```
## POA&M (Plan of Action and Milestones) Tracking
```yaml
# poam_template.yaml
poam_entry:
- id: "POAM-2025-001"
weakness: "AC-2(3) - Automated account disable after 90 days inactivity not implemented"
control: "AC-2"
risk_level: "moderate"
finding_source: "3PAO Annual Assessment - 2025"
date_identified: "2025-03-15"
scheduled_completion: "2025-06-15"
milestone_1:
description: "Configure IdP inactivity policy"
target_date: "2025-04-15"
status: "complete"
milestone_2:
description: "Test automated disable in staging"
target_date: "2025-05-01"
status: "in_progress"
milestone_3:
description: "Deploy to production and validate"
target_date: "2025-06-15"
status: "not_started"
responsible_party: "IAM Team"
status: "open"
vendor_dependency: false
- id: "POAM-2025-002"
weakness: "RA-5 - Vulnerability scan coverage does not include container images"
control: "RA-5"
risk_level: "high"
finding_source: "3PAO Annual Assessment - 2025"
date_identified: "2025-03-15"
scheduled_completion: "2025-05-15"
milestone_1:
description: "Evaluate and select container scanning tool"
target_date: "2025-04-01"
status: "complete"
milestone_2:
description: "Integrate scanning into CI/CD pipeline"
target_date: "2025-04-30"
status: "in_progress"
milestone_3:
description: "Demonstrate full coverage to 3PAO"
target_date: "2025-05-15"
status: "not_started"
responsible_party: "Security Engineering"
status: "open"
vendor_dependency: false
poam_aging_thresholds:
high: "Must be resolved within 30 days"
moderate: "Must be resolved within 90 days"
low: "Must be resolved within 180 days"
overdue_escalation: "Reported to authorizing official monthly"
```
## Continuous Monitoring (ConMon) Procedures
```yaml
continuous_monitoring:
monthly:
vulnerability_scanning:
scope: "All operating systems, databases, web applications, and containers"
tool: "Tenable.io, Qualys, or equivalent"
deliverable: "Monthly scan report with remediation status"
sla:
critical_cvss_9_plus: "Remediate within 30 days"
high_cvss_7_to_9: "Remediate within 30 days"
moderate_cvss_4_to_7: "Remediate within 90 days"
low_cvss_below_4: "Remediate within 180 days"
poam_updates:
action: "Update all open POA&M items with current status"
deliverable: "Updated POA&M spreadsheet submitted to agency"
content:
- "Milestone completion updates"
- "New POA&M items from scans"
- "Closed POA&M items with evidence"
inventory_updates:
action: "Review and update system component inventory"
deliverable: "Updated hardware and software inventory"
quarterly:
- "Review and update SSP with any system changes"
- "Submit ConMon deliverables package to agency"
- "Review access control lists and user accounts"
- "Update network diagrams if changes occurred"
annual:
security_assessment:
performed_by: "3PAO"
scope: "Subset of controls (~1/3 each year, full coverage in 3 years)"
deliverable: "Security Assessment Report (SAR)"
penetration_testing:
performed_by: "3PAO or qualified third party"
scope: "External and internal network, web applications"
deliverable: "Penetration test report with findings"
contingency_plan_test:
scope: "Full DR/BCP test including failover"
deliverable: "Contingency plan test report"
incident_response_test:
scope: "Tabletop exercise or functional exercise"
deliverable: "IR test report with lessons learned"
```
## FedRAMP FIPS 140-2 Cryptography Requirements
```bash
# Verify FIPS mode is enabled on Linux systems
cat /proc/sys/crypto/fips_enabled
# Output should be: 1
# Check OpenSSL FIPS module
openssl version
openssl list -providers # Should show FIPS provider
# AWS: Use FIPS endpoints
# Example: Use FIPS endpoint for S3
aws s3 ls --endpoint-url https://s3-fips.us-east-1.amazonaws.com
# Configure AWS CLI for FIPS
# ~/.aws/config
# [default]
# use_fips_endpoint = true
# Verify TLS configuration meets FedRAMP requirements
openssl s_client -connect your-service.example.com:443 -tls1_2 < /dev/null 2>/dev/null | \
grep -E "Protocol|Cipher"
# Must be TLS 1.2 or higher with FIPS-approved cipher suites
```
## FedRAMP Authorization Checklist
```yaml
authorization_checklist:
pre_authorization:
- [ ] Determine impact level (Low, Moderate, High)
- [ ] Choose authorization path (Agency ATO or JAB P-ATO)
- [ ] Engage FedRAMP PMO for readiness assessment
- [ ] Select 3PAO from FedRAMP marketplace
- [ ] Complete SSP with all control implementations documented
- [ ] Develop required policies and procedures
- [ ] Implement all applicable NIST 800-53 controls
- [ ] Ensure FIPS 140-2 validated cryptographic modules in use
assessment:
- [ ] 3PAO conducts readiness assessment (optional but recommended)
- [ ] 3PAO conducts full security assessment
- [ ] 3PAO delivers Security Assessment Report (SAR)
- [ ] Develop POA&M for all findings
- [ ] Remediate critical and high findings before authorization
authorization_package:
- [ ] System Security Plan (SSP)
- [ ] Security Assessment Report (SAR)
- [ ] Plan of Action and Milestones (POA&M)
- [ ] Continuous Monitoring Plan
- [ ] Incident Response Plan
- [ ] Contingency Plan
- [ ] Configuration Management Plan
- [ ] Control Implementation Summary (CIS)
- [ ] Interconnection Security Agreements
post_authorization:
- [ ] Establish ConMon program with monthly deliverables
- [ ] Monthly vulnerability scanning and POA&M updates
- [ ] Annual 3PAO assessment of control subset
- [ ] Annual penetration testing
- [ ] Report significant changes to authorizing official
- [ ] Report security incidents to US-CERT within 1 hour
- [ ] Maintain authorization by meeting ConMon requirements
```
## Best Practices
- 3PAO assessment
- SSP documentation
- POA&M tracking
- Continuous monitoring
- Annual authorization
- Start with a FedRAMP Readiness Assessment to identify gaps before the formal 3PAO assessment
- Use the FedRAMP SSP template exactly as provided to avoid review delays
- Inherit controls from your IaaS provider (AWS GovCloud, Azure Government) and document the inheritance clearly
- Implement FIPS 140-2 validated cryptographic modules for all encryption (TLS, at-rest, key management)
- Automate continuous monitoring deliverables to reduce manual effort and human error
- Maintain POA&M items within aging thresholds; overdue items risk losing authorization
- Report significant system changes to the authorizing official before implementation
- Treat the SSP as a living document and update it with every change to the system boundary
- Use US-CERT reporting procedures and maintain the 1-hour incident notification requirement
- Engage the FedRAMP PMO early and often for guidance on the authorization process
compliance/frameworks/gdpr-compliance/SKILL.md
+533 -37
View File
@@ -9,56 +9,552 @@ metadata:
# GDPR Compliance
Implement GDPR requirements for EU data protection.
Implement General Data Protection Regulation requirements for organizations that process personal data of EU/EEA residents, covering lawful processing, data subject rights, and technical safeguards.
## Key Principles
## When to Use
- Processing personal data of EU/EEA residents in any capacity
- Building consent management and preference centers
- Implementing Data Subject Access Request (DSAR) workflows
- Conducting Data Protection Impact Assessments (DPIAs)
- Setting up data processing agreements with third-party processors
- Designing systems with privacy by design and by default principles
## Key Principles and Legal Bases
```yaml
principles:
lawfulness: Legal basis for processing
purpose_limitation: Specific, explicit purposes
data_minimization: Adequate, relevant, limited
accuracy: Accurate and up to date
storage_limitation: No longer than necessary
integrity: Secure processing
accountability: Demonstrate compliance
gdpr_principles:
article_5:
lawfulness_fairness_transparency:
description: "Process data lawfully, fairly, and transparently"
implementation:
- Document legal basis for every processing activity
- Provide clear privacy notices
- No hidden or deceptive data collection
purpose_limitation:
description: "Collect for specified, explicit, and legitimate purposes"
implementation:
- Define purpose before collection
- Do not repurpose data without new legal basis
- Document all processing purposes in ROPA
data_minimization:
description: "Adequate, relevant, and limited to what is necessary"
implementation:
- Collect only required fields
- Review data models for unnecessary fields
- Remove optional fields that are not used
accuracy:
description: "Accurate and kept up to date"
implementation:
- Provide self-service profile editing
- Implement data validation at point of entry
- Schedule regular data quality reviews
storage_limitation:
description: "Kept no longer than necessary"
implementation:
- Define retention periods per data category
- Automate deletion when retention expires
- Document retention schedule
integrity_and_confidentiality:
description: "Appropriate security measures"
implementation:
- Encryption at rest and in transit
- Access controls and audit logging
- Pseudonymization where appropriate
accountability:
description: "Demonstrate compliance"
implementation:
- Maintain Records of Processing Activities
- Conduct DPIAs for high-risk processing
- Appoint DPO if required
legal_bases:
article_6:
consent: "Freely given, specific, informed, unambiguous"
contract: "Necessary for performance of a contract"
legal_obligation: "Required by EU or member state law"
vital_interests: "Protect life of data subject or another person"
public_interest: "Task carried out in public interest"
legitimate_interest: "Legitimate interest not overridden by data subject rights"
```
## Data Subject Rights
## Data Mapping Template (Records of Processing Activities)
```yaml
rights:
- Right to access
- Right to rectification
- Right to erasure
- Right to restrict processing
- Right to data portability
- Right to object
- Rights related to automated decisions
# Record of Processing Activities (ROPA) - Article 30
processing_activity:
name: "Customer Account Management"
controller: "Example Corp, 123 Main St, Dublin, Ireland"
dpo_contact: "dpo@example.com"
purpose: "Manage customer accounts, provide services, handle billing"
legal_basis: "Contract (Art. 6(1)(b))"
categories_of_data_subjects:
- Customers
- Prospective customers
categories_of_personal_data:
- Name, email, phone number
- Billing address
- Payment information (tokenized)
- Service usage data
- Support ticket history
special_categories: "None"
recipients:
- Payment processor (Stripe) - processor
- Email service (SendGrid) - processor
- Cloud hosting (AWS) - processor
international_transfers:
- Destination: United States
Safeguard: "Standard Contractual Clauses (SCCs)"
TIA_completed: true
retention_period: "Account data retained for duration of contract + 7 years for legal obligations"
security_measures:
- AES-256 encryption at rest
- TLS 1.3 in transit
- Role-based access control
- Audit logging of all access
dpia_required: false
last_reviewed: "2024-06-01"
# Template for each processing activity
processing_activity_template:
name: ""
controller: ""
joint_controller: "" # if applicable
processor: "" # if acting as processor
dpo_contact: ""
purpose: ""
legal_basis: "" # consent | contract | legal_obligation | vital_interests | public_interest | legitimate_interest
legitimate_interest_assessment: "" # if legitimate interest
categories_of_data_subjects: []
categories_of_personal_data: []
special_categories: "" # Art. 9 data
recipients: []
international_transfers: []
retention_period: ""
security_measures: []
dpia_required: false
date_added: ""
last_reviewed: ""
```
## Technical Implementation
## Consent Management Implementation
```python
# Data export for portability
def export_user_data(user_id):
return {
"profile": get_profile(user_id),
"activity": get_activity_log(user_id),
"preferences": get_preferences(user_id)
}
"""
Consent management system implementing GDPR Article 7 requirements.
Consent must be freely given, specific, informed, and unambiguous.
"""
from datetime import datetime, timezone
from enum import Enum
import json
import hashlib
# Right to erasure
def delete_user_data(user_id):
anonymize_profile(user_id)
delete_activity_log(user_id)
log_deletion(user_id)
class ConsentPurpose(Enum):
MARKETING_EMAIL = "marketing_email"
MARKETING_SMS = "marketing_sms"
ANALYTICS = "analytics"
PERSONALIZATION = "personalization"
THIRD_PARTY_SHARING = "third_party_sharing"
PROFILING = "profiling"
class ConsentManager:
def __init__(self, db):
self.db = db
def record_consent(self, user_id, purpose, granted, source,
privacy_policy_version, ip_address=None):
"""Record a consent decision with full audit trail."""
consent_record = {
"user_id": user_id,
"purpose": purpose.value,
"granted": granted,
"timestamp": datetime.now(timezone.utc).isoformat(),
"source": source, # e.g., "web_signup", "preference_center", "cookie_banner"
"privacy_policy_version": privacy_policy_version,
"ip_address": ip_address,
"withdrawal_timestamp": None,
}
# Store with immutable audit trail
consent_record["record_hash"] = hashlib.sha256(
json.dumps(consent_record, sort_keys=True).encode()
).hexdigest()
self.db.consent_records.insert(consent_record)
return consent_record
def withdraw_consent(self, user_id, purpose):
"""Process consent withdrawal - must be as easy as giving consent."""
record = self.record_consent(
user_id=user_id,
purpose=purpose,
granted=False,
source="withdrawal",
privacy_policy_version="N/A",
)
# Trigger downstream actions
self._notify_processors(user_id, purpose, "withdrawn")
self._stop_processing(user_id, purpose)
return record
def get_consent_status(self, user_id, purpose):
"""Get current consent status for a specific purpose."""
latest = self.db.consent_records.find_one(
{"user_id": user_id, "purpose": purpose.value},
sort=[("timestamp", -1)]
)
return latest["granted"] if latest else False
def get_all_consents(self, user_id):
"""Get all consent records for a user (for DSAR response)."""
return list(self.db.consent_records.find(
{"user_id": user_id},
sort=[("timestamp", -1)]
))
def export_consent_proof(self, user_id, purpose):
"""Export verifiable consent proof for accountability."""
records = list(self.db.consent_records.find(
{"user_id": user_id, "purpose": purpose.value},
sort=[("timestamp", 1)]
))
return {
"user_id": user_id,
"purpose": purpose.value,
"consent_history": records,
"current_status": self.get_consent_status(user_id, purpose),
"exported_at": datetime.now(timezone.utc).isoformat(),
}
def _notify_processors(self, user_id, purpose, action):
"""Notify downstream processors of consent change."""
pass # Implement webhook/API calls to processors
def _stop_processing(self, user_id, purpose):
"""Immediately stop processing for withdrawn consent."""
pass # Implement processing halt logic
```
## Data Subject Access Request (DSAR) Procedures
```yaml
dsar_workflow:
step_1_receive:
actions:
- Log the request with timestamp and channel received
- Assign unique tracking ID
- Acknowledge receipt within 3 business days
identity_verification:
- Verify identity before providing any data
- Use existing authentication where possible
- Request additional proof if necessary (but not excessive)
sla: "Must respond within 30 days (extendable to 90 days for complex requests)"
step_2_assess:
actions:
- Determine request type (access, rectification, erasure, portability, etc.)
- Identify all systems containing the individual's data
- Check for lawful grounds to refuse (legal obligations, etc.)
- Assess if extension is needed (complex or numerous requests)
step_3_collect:
systems_to_search:
- Primary application database
- CRM system
- Email marketing platform
- Analytics systems
- Customer support tickets
- Backup systems (if practically retrievable)
- Log files containing PII
- Third-party processors (request from each)
step_4_respond:
access_request:
- Provide copy of all personal data in commonly used electronic format
- Include processing purposes, categories, recipients, retention periods
- Include source of data if not collected from the individual
- Include information about automated decision-making
rectification_request:
- Update data in all systems
- Notify all recipients of the correction
erasure_request:
- Delete data from all active systems
- Remove from backups where technically feasible
- Notify all processors and recipients
- Document what was deleted and any retained data with legal basis
portability_request:
- Provide data in structured, machine-readable format (JSON/CSV)
- Include only data provided by the data subject
- Transfer directly to another controller if requested and feasible
step_5_close:
actions:
- Send response to data subject
- Document the entire handling process
- Archive DSAR record for accountability
- Update data mapping if new data stores discovered
```
```python
"""DSAR automation - data collection across systems."""
import json
from datetime import datetime, timezone
class DSARProcessor:
def __init__(self, data_sources):
self.data_sources = data_sources # Dict of system_name: DataSource
def process_access_request(self, user_identifier):
"""Collect all personal data across registered systems."""
collected_data = {
"request_id": f"DSAR-{datetime.now(timezone.utc).strftime('%Y%m%d%H%M%S')}",
"generated_at": datetime.now(timezone.utc).isoformat(),
"data_subject": user_identifier,
"systems": {},
}
for system_name, source in self.data_sources.items():
try:
data = source.extract_user_data(user_identifier)
collected_data["systems"][system_name] = {
"status": "collected",
"record_count": len(data) if isinstance(data, list) else 1,
"data": data,
}
except Exception as e:
collected_data["systems"][system_name] = {
"status": "error",
"error": str(e),
}
return collected_data
def process_erasure_request(self, user_identifier):
"""Delete personal data across all systems (right to erasure)."""
results = {
"request_id": f"ERASE-{datetime.now(timezone.utc).strftime('%Y%m%d%H%M%S')}",
"data_subject": user_identifier,
"systems": {},
}
for system_name, source in self.data_sources.items():
try:
deleted = source.delete_user_data(user_identifier)
retained = source.get_retained_data(user_identifier)
results["systems"][system_name] = {
"status": "deleted",
"records_deleted": deleted,
"retained_data": retained, # Data kept for legal obligations
"retention_basis": source.retention_legal_basis,
}
except Exception as e:
results["systems"][system_name] = {
"status": "error",
"error": str(e),
}
return results
def export_portable_data(self, user_identifier, format="json"):
"""Export data in machine-readable format for portability."""
data = self.process_access_request(user_identifier)
if format == "json":
return json.dumps(data, indent=2, default=str)
elif format == "csv":
return self._convert_to_csv(data)
raise ValueError(f"Unsupported format: {format}")
```
## Data Processing Agreement (DPA) Requirements
```yaml
dpa_requirements:
mandatory_clauses:
article_28:
- Subject matter, duration, nature, and purpose of processing
- Type of personal data and categories of data subjects
- Obligations and rights of the controller
- Processing only on documented instructions from controller
- Confidentiality obligations on processor personnel
- Appropriate technical and organizational security measures
- Conditions for engaging sub-processors (prior authorization)
- Assistance with data subject rights requests
- Assistance with security obligations (Art. 32-36)
- Deletion or return of data after service ends
- Audit and inspection rights for the controller
sub_processor_management:
- [ ] List of current sub-processors provided by processor
- [ ] Notification mechanism for new sub-processors (30-day notice)
- [ ] Right to object to new sub-processors
- [ ] Sub-processors bound by same data protection obligations
- [ ] Processor remains liable for sub-processor compliance
international_transfers:
mechanisms:
- Standard Contractual Clauses (SCCs) - most common
- Binding Corporate Rules (BCRs) - intra-group transfers
- Adequacy decision (countries deemed adequate by EC)
- Derogations for specific situations (explicit consent, contract necessity)
transfer_impact_assessment:
- [ ] Assess laws of the destination country
- [ ] Evaluate effectiveness of safeguards
- [ ] Document supplementary measures if needed
- [ ] Review periodically for legal changes
dpa_registry:
track_per_processor:
- Processor name and contact details
- DPA execution date
- Data types processed
- Sub-processors and their locations
- SCC version used for international transfers
- TIA completion date
- Next review date
```
## Data Protection Impact Assessment (DPIA) Template
```yaml
dpia_template:
when_required:
- Systematic and extensive profiling with significant effects
- Large-scale processing of special category data
- Systematic monitoring of publicly accessible areas
- Any processing on national supervisory authority's list
- New technologies with likely high risk to rights and freedoms
assessment:
section_1_description:
processing_activity: ""
purpose: ""
legal_basis: ""
data_categories: []
data_subjects: []
recipients: []
retention: ""
data_flows: "Describe how data moves through systems"
section_2_necessity:
is_processing_necessary: ""
is_processing_proportionate: ""
alternatives_considered: ""
data_minimization_applied: ""
section_3_risks:
risk_assessment:
- risk: "Unauthorized access to personal data"
likelihood: "medium"
severity: "high"
risk_level: "high"
existing_controls: "Encryption, access controls, audit logs"
residual_risk: "medium"
- risk: "Accidental data loss or destruction"
likelihood: "low"
severity: "high"
risk_level: "medium"
existing_controls: "Backups, replication, DR procedures"
residual_risk: "low"
- risk: "Excessive data collection beyond purpose"
likelihood: "medium"
severity: "medium"
risk_level: "medium"
existing_controls: "Data minimization review, schema validation"
residual_risk: "low"
section_4_measures:
technical_measures:
- Pseudonymization of personal data
- Encryption at rest (AES-256) and in transit (TLS 1.3)
- Access controls with least privilege
- Automated data retention enforcement
organizational_measures:
- Staff training on data protection
- Data protection policies and procedures
- Incident response procedures
- Regular access reviews
monitoring:
- Audit logging of all data access
- Anomaly detection for unusual access patterns
- Regular compliance testing
section_5_sign_off:
dpo_consultation: "Required if high residual risk"
dpo_opinion: ""
supervisory_authority_consultation: "Required if risk cannot be mitigated"
approval_date: ""
next_review_date: ""
```
## GDPR Compliance Checklist
```yaml
gdpr_compliance_checklist:
governance:
- [ ] Data Protection Officer appointed (if required under Art. 37)
- [ ] Records of Processing Activities (ROPA) maintained
- [ ] Privacy policies published and up to date
- [ ] Data protection training conducted for all staff
- [ ] Data breach response plan documented and tested
lawful_processing:
- [ ] Legal basis identified and documented for each processing activity
- [ ] Consent mechanisms comply with Art. 7 (freely given, specific, informed)
- [ ] Consent withdrawal is as easy as giving consent
- [ ] Legitimate interest assessments completed where applicable
- [ ] Special category data has Art. 9 legal basis documented
data_subject_rights:
- [ ] DSAR intake process established (multiple channels)
- [ ] Identity verification procedure defined
- [ ] Response within 30 days (or extension communicated)
- [ ] Right to access implemented and tested
- [ ] Right to rectification implemented
- [ ] Right to erasure implemented with legal retention exceptions
- [ ] Right to portability implemented (structured, machine-readable export)
- [ ] Right to object implemented (especially for direct marketing)
technical_measures:
- [ ] Encryption at rest and in transit for all personal data
- [ ] Pseudonymization applied where feasible
- [ ] Access controls enforce least privilege
- [ ] Audit logging of personal data access
- [ ] Data retention automated with defined schedules
- [ ] Secure deletion procedures verified
third_parties:
- [ ] Data Processing Agreements signed with all processors
- [ ] Sub-processor notification mechanism in place
- [ ] International transfer safeguards implemented (SCCs, etc.)
- [ ] Transfer Impact Assessments completed
- [ ] Processor compliance verified periodically
breach_management:
- [ ] Breach detection and assessment procedures documented
- [ ] 72-hour supervisory authority notification process ready
- [ ] Individual notification procedures for high-risk breaches
- [ ] Breach register maintained
- [ ] Post-breach review and improvement process
```
## Best Practices
- Privacy impact assessments
- Data processing agreements
- Consent management
- Breach notification (72 hours)
- Data Protection Officer (if required)
- Maintain a comprehensive Records of Processing Activities as the foundation of GDPR compliance
- Implement privacy by design: build data protection into systems from the start, not retrofitted
- Apply data minimization rigorously: do not collect personal data "just in case"
- Automate DSAR processing to meet the 30-day response deadline consistently
- Keep consent granular and purpose-specific; avoid bundled consent for multiple purposes
- Conduct DPIAs before launching high-risk processing activities
- Ensure data processing agreements are signed with every processor before sharing personal data
- Implement automated retention enforcement to prevent storage beyond defined periods
- Train all staff who handle personal data, not just the IT and legal teams
- Regularly audit data flows to discover shadow processing or undocumented data stores
compliance/frameworks/hipaa-compliance/SKILL.md
+400 -46
View File
@@ -9,66 +9,420 @@ metadata:
# HIPAA Compliance
Implement HIPAA requirements for healthcare data protection.
Implement HIPAA Security Rule, Privacy Rule, and Breach Notification Rule requirements for systems that create, receive, maintain, or transmit electronic Protected Health Information (ePHI).
## HIPAA Rules
## When to Use
- Building or operating systems that handle electronic Protected Health Information
- Configuring cloud infrastructure for HIPAA-eligible workloads
- Establishing Business Associate Agreements with vendors
- Implementing technical safeguards for PHI protection
- Preparing for HIPAA compliance audits or OCR investigations
## HIPAA Rules and Safeguards
```yaml
security_rule:
administrative:
- Risk analysis
- Security management
- Workforce training
- Contingency planning
physical:
- Facility access
- Workstation security
- Device controls
technical:
- Access control
- Audit controls
- Integrity controls
- Transmission security
administrative_safeguards:
164.308_a_1: "Security Management Process"
actions:
- Conduct risk analysis (required)
- Implement risk management program (required)
- Apply sanction policy for violations (required)
- Review information system activity (required)
164.308_a_3: "Workforce Security"
actions:
- Authorization/supervision procedures (addressable)
- Workforce clearance procedure (addressable)
- Termination procedures (addressable)
164.308_a_4: "Information Access Management"
actions:
- Access authorization policies (addressable)
- Access establishment and modification (addressable)
- Isolate healthcare clearinghouse functions (required)
164.308_a_5: "Security Awareness and Training"
actions:
- Security reminders (addressable)
- Protection from malicious software (addressable)
- Log-in monitoring (addressable)
- Password management (addressable)
164.308_a_6: "Security Incident Procedures"
actions:
- Response and reporting procedures (required)
164.308_a_7: "Contingency Plan"
actions:
- Data backup plan (required)
- Disaster recovery plan (required)
- Emergency mode operation plan (required)
- Testing and revision procedures (addressable)
- Applications and data criticality analysis (addressable)
164.308_a_8: "Evaluation"
actions:
- Periodic technical and nontechnical evaluation (required)
physical_safeguards:
164.310_a: "Facility Access Controls"
164.310_b: "Workstation Use"
164.310_c: "Workstation Security"
164.310_d: "Device and Media Controls"
technical_safeguards:
164.312_a: "Access Control"
actions:
- Unique user identification (required)
- Emergency access procedure (required)
- Automatic logoff (addressable)
- Encryption and decryption (addressable)
164.312_b: "Audit Controls"
actions:
- Implement hardware/software/procedural mechanisms to record and examine access (required)
164.312_c: "Integrity"
actions:
- Mechanism to authenticate ePHI (addressable)
164.312_d: "Person or Entity Authentication"
actions:
- Verify identity of person/entity seeking access (required)
164.312_e: "Transmission Security"
actions:
- Integrity controls (addressable)
- Encryption (addressable)
privacy_rule:
minimum_necessary: "Limit PHI use, disclosure, and requests to minimum necessary"
individual_rights: "Access, amendment, accounting of disclosures, restrictions"
notice_of_practices: "Provide notice of privacy practices to individuals"
breach_notification_rule:
individual_notification: "Within 60 days of discovery"
hhs_notification: "Annual for <500 records; within 60 days for 500+"
media_notification: "Required when 500+ individuals in a state/jurisdiction"
```
## Technical Safeguards
## Technical Safeguards Implementation Checklist
```yaml
requirements:
encryption:
at_rest: AES-256
in_transit: TLS 1.2+
access_control:
- Unique user IDs
- Emergency access procedure
- Automatic logoff
- Encryption/decryption
audit:
- Access logging
- Activity monitoring
- Log retention (6 years)
encryption_requirements:
at_rest:
standard: AES-256
aws_services:
- [ ] RDS encryption enabled (KMS managed key)
- [ ] S3 bucket default encryption (SSE-KMS)
- [ ] EBS volume encryption enabled
- [ ] DynamoDB table encryption (KMS)
- [ ] ElastiCache encryption at rest enabled
- [ ] Redshift cluster encryption enabled
- [ ] EFS encryption enabled
azure_services:
- [ ] Azure SQL TDE enabled (customer-managed key)
- [ ] Storage Account encryption (CMK)
- [ ] Managed Disk encryption (SSE with CMK)
- [ ] Cosmos DB encryption at rest
gcp_services:
- [ ] Cloud SQL encryption (CMEK)
- [ ] Cloud Storage encryption (CMEK)
- [ ] BigQuery encryption (CMEK)
- [ ] Persistent Disk encryption (CMEK)
in_transit:
standard: TLS 1.2 or higher
checks:
- [ ] TLS 1.2+ enforced on all load balancers
- [ ] HTTP-to-HTTPS redirect enabled
- [ ] Internal service-to-service mTLS configured
- [ ] Database connections use SSL/TLS
- [ ] API gateways enforce TLS minimum version
- [ ] Email encryption for PHI (S/MIME or TLS)
- [ ] VPN or private connectivity for admin access
key_management:
- [ ] Customer-managed KMS keys for PHI data stores
- [ ] Key rotation enabled (annual minimum)
- [ ] Key access restricted to authorized roles only
- [ ] Key usage audited via CloudTrail / audit logs
- [ ] Key deletion protection enabled
access_control:
unique_user_identification:
- [ ] Individual user accounts (no shared credentials)
- [ ] MFA enforced for all users accessing PHI systems
- [ ] Service accounts with unique identities and audited usage
- [ ] Federated identity with SSO (SAML/OIDC)
role_based_access:
- [ ] Least privilege roles defined per job function
- [ ] PHI access restricted to need-to-know
- [ ] Separate roles for data access vs. administration
- [ ] Privileged access requires just-in-time approval
session_management:
- [ ] Automatic session timeout (15 minutes idle for workstations)
- [ ] Re-authentication for sensitive operations
- [ ] Concurrent session limits
- [ ] Session tokens secured (HttpOnly, Secure, SameSite)
emergency_access:
- [ ] Break-glass procedure documented and tested
- [ ] Emergency access credentials stored securely
- [ ] All emergency access usage audited and reviewed
- [ ] Emergency access automatically expires
audit_controls:
logging_requirements:
- [ ] All PHI access logged (read, write, delete)
- [ ] User authentication events logged
- [ ] Administrative actions logged
- [ ] Failed access attempts logged
- [ ] Log integrity protection (hash chaining or WORM storage)
- [ ] Logs retained for minimum 6 years
- [ ] Regular log review process documented
monitoring:
- [ ] Real-time alerting on unauthorized PHI access attempts
- [ ] Anomaly detection for unusual data access patterns
- [ ] Privileged action monitoring
- [ ] Data export/download alerting
```
## AWS HIPAA Setup
## AWS HIPAA-Eligible Architecture
```bash
# Enable CloudTrail for HIPAA auditing
aws cloudtrail create-trail \
--name hipaa-audit-trail \
--s3-bucket-name hipaa-logs \
--is-multi-region-trail \
--enable-log-file-validation
# Verify you are using only HIPAA-eligible AWS services
# Reference: https://aws.amazon.com/compliance/hipaa-eligible-services-reference/
# Use HIPAA-eligible services only
# Create a dedicated VPC for PHI workloads
aws ec2 create-vpc --cidr-block 10.100.0.0/16 \
--tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=phi-vpc},{Key=Compliance,Value=HIPAA}]'
# Enable VPC flow logs for network auditing
aws ec2 create-flow-log \
--resource-type VPC \
--resource-ids vpc-XXXXXXXX \
--traffic-type ALL \
--log-destination-type cloud-watch-logs \
--log-group-name /vpc/phi-flow-logs \
--deliver-logs-permission-arn arn:aws:iam::123456789012:role/VPCFlowLogsRole
# Create encrypted RDS instance for PHI
aws rds create-db-instance \
--db-instance-identifier phi-database \
--db-instance-class db.r6g.large \
--engine postgres \
--master-username admin \
--master-user-password "USE_SECRETS_MANAGER" \
--storage-encrypted \
--kms-key-id arn:aws:kms:us-east-1:123456789012:alias/phi-rds-key \
--vpc-security-group-ids sg-XXXXXXXX \
--db-subnet-group-name phi-subnet-group \
--backup-retention-period 35 \
--multi-az \
--deletion-protection \
--enable-cloudwatch-logs-exports '["postgresql","upgrade"]' \
--tags Key=Compliance,Value=HIPAA Key=DataClassification,Value=PHI
# Create S3 bucket with HIPAA controls
aws s3api create-bucket --bucket phi-data-bucket --region us-east-1
aws s3api put-bucket-encryption --bucket phi-data-bucket \
--server-side-encryption-configuration '{
"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "aws:kms", "KMSMasterKeyID": "alias/phi-s3-key"}, "BucketKeyEnabled": true}]
}'
aws s3api put-public-access-block --bucket phi-data-bucket \
--public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
aws s3api put-bucket-versioning --bucket phi-data-bucket \
--versioning-configuration Status=Enabled
aws s3api put-bucket-logging --bucket phi-data-bucket \
--bucket-logging-status '{"LoggingEnabled": {"TargetBucket": "phi-access-logs", "TargetPrefix": "phi-data-bucket/"}}'
# Enable CloudTrail data events for PHI buckets
aws cloudtrail put-event-selectors --trail-name hipaa-audit-trail \
--advanced-event-selectors '[{
"Name": "PHI-S3-DataEvents",
"FieldSelectors": [
{"Field": "eventCategory", "Equals": ["Data"]},
{"Field": "resources.type", "Equals": ["AWS::S3::Object"]},
{"Field": "resources.ARN", "StartsWith": ["arn:aws:s3:::phi-data-bucket/"]}
]
}]'
```
## Business Associate Agreement Tracking
```yaml
baa_tracking:
required_when:
- Vendor creates, receives, maintains, or transmits PHI on your behalf
- Subcontractor of a business associate handles PHI
- Cloud service provider stores or processes PHI
not_required_for:
- Conduit exception (postal service, ISP carrying encrypted data)
- Treatment providers sharing PHI for treatment purposes
- Plan sponsor receiving summary health information
baa_registry:
format:
vendor_name: ""
baa_execution_date: ""
baa_expiration_date: ""
phi_types_shared: []
services_provided: ""
subcontractors_identified: []
breach_notification_sla: "hours"
last_risk_assessment: ""
next_review_date: ""
status: "active | pending | expired"
cloud_provider_baas:
aws:
- Sign AWS BAA via AWS Artifact in the console
- Applies to all HIPAA-eligible services in the account
- Must restrict PHI to eligible services only
azure:
- Microsoft BAA is part of Online Services Terms
- Automatically applies when using qualifying services
gcp:
- Sign Google Cloud BAA via Google Workspace Admin or Cloud console
- Covers HIPAA-eligible GCP services
review_schedule:
- [ ] Annual review of all active BAAs
- [ ] Verify vendor compliance certifications are current
- [ ] Confirm subcontractor BAAs are in place
- [ ] Update BAA registry with any vendor changes
- [ ] Assess vendor security posture annually
```
## Risk Analysis Automation
```bash
#!/usr/bin/env bash
# hipaa-risk-scan.sh - Technical risk analysis checks for HIPAA
echo "=== HIPAA Technical Safeguard Checks ==="
echo "--- Encryption at Rest ---"
# Check for unencrypted RDS instances
UNENCRYPTED_RDS=$(aws rds describe-db-instances \
--query 'DBInstances[?StorageEncrypted==`false`].DBInstanceIdentifier' --output text)
[ -z "$UNENCRYPTED_RDS" ] && echo "PASS: All RDS instances encrypted" || \
echo "FAIL: Unencrypted RDS: $UNENCRYPTED_RDS"
# Check for unencrypted S3 buckets
for bucket in $(aws s3api list-buckets --query 'Buckets[*].Name' --output text); do
enc=$(aws s3api get-bucket-encryption --bucket "$bucket" 2>/dev/null)
[ -z "$enc" ] && echo "FAIL: S3 bucket $bucket has no default encryption"
done
# Check for unencrypted EBS volumes
UNENCRYPTED_EBS=$(aws ec2 describe-volumes \
--query 'Volumes[?Encrypted==`false`].VolumeId' --output text)
[ -z "$UNENCRYPTED_EBS" ] && echo "PASS: All EBS volumes encrypted" || \
echo "FAIL: Unencrypted EBS: $UNENCRYPTED_EBS"
echo "--- Access Control ---"
# Check for users without MFA
aws iam generate-credential-report > /dev/null 2>&1 && sleep 5
aws iam get-credential-report --output text --query Content | base64 -d | \
awk -F, '$4=="true" && $8=="false" {print "FAIL: User without MFA: "$1}'
# Check for unused access keys (90+ days)
THRESHOLD=$(date -d '90 days ago' +%Y-%m-%dT%H:%M:%S 2>/dev/null || date -v-90d +%Y-%m-%dT%H:%M:%S)
aws iam get-credential-report --output text --query Content | base64 -d | \
awk -F, -v t="$THRESHOLD" 'NR>1 && $11!="N/A" && $11<t {print "WARN: Stale access key for "$1}'
echo "--- Audit Controls ---"
# Verify CloudTrail is logging
CT_STATUS=$(aws cloudtrail get-trail-status --name hipaa-audit-trail --query 'IsLogging' --output text)
[ "$CT_STATUS" = "True" ] && echo "PASS: CloudTrail active" || echo "FAIL: CloudTrail not logging"
# Verify VPC flow logs
for vpc in $(aws ec2 describe-vpcs --query 'Vpcs[*].VpcId' --output text); do
fl=$(aws ec2 describe-flow-logs --filter "Name=resource-id,Values=$vpc" --query 'FlowLogs[0].FlowLogId' --output text)
[ "$fl" = "None" ] && echo "FAIL: No flow logs for VPC $vpc"
done
echo "--- Transmission Security ---"
# Check for ALBs without HTTPS listener
for alb in $(aws elbv2 describe-load-balancers --query 'LoadBalancers[*].LoadBalancerArn' --output text); do
HTTPS=$(aws elbv2 describe-listeners --load-balancer-arn "$alb" \
--query 'Listeners[?Protocol==`HTTPS`].ListenerArn' --output text)
[ -z "$HTTPS" ] && echo "FAIL: ALB without HTTPS: $alb"
done
echo "=== Scan complete ==="
```
## HIPAA Compliance Checklist
```yaml
hipaa_compliance_checklist:
administrative:
- [ ] Risk analysis conducted and documented
- [ ] Risk management plan implemented
- [ ] Security officer designated
- [ ] Privacy officer designated
- [ ] Workforce security awareness training completed
- [ ] Sanction policy documented and communicated
- [ ] Contingency plan (backup, DR, emergency mode) documented
- [ ] Business associate agreements signed for all applicable vendors
- [ ] Periodic evaluation/audit scheduled
technical:
- [ ] Unique user identification enforced
- [ ] MFA enabled for all PHI system access
- [ ] Automatic logoff configured (15-minute timeout)
- [ ] Encryption at rest (AES-256) for all PHI stores
- [ ] Encryption in transit (TLS 1.2+) for all PHI transmission
- [ ] Audit logging enabled for all PHI access
- [ ] Log retention configured for 6+ years
- [ ] Integrity controls on PHI (checksums, signatures)
- [ ] Emergency access (break-glass) procedure tested
physical:
- [ ] Facility access controls documented
- [ ] Workstation use policy in place
- [ ] Device and media disposal procedures documented
- [ ] Media re-use procedures documented
breach_response:
- [ ] Breach notification procedure documented
- [ ] Breach risk assessment methodology defined
- [ ] Individual notification template prepared
- [ ] HHS notification process understood
- [ ] Breach log maintained
- [ ] Annual breach assessment reviewed
operational:
- [ ] PHI data inventory maintained
- [ ] Minimum necessary access enforced
- [ ] Access reviews conducted quarterly
- [ ] Vendor risk assessments current
- [ ] Incident response plan tested annually
- [ ] Policies reviewed and updated annually
```
## Best Practices
- Business Associate Agreements (BAAs)
- Minimum necessary access
- Breach notification procedures
- Regular risk assessments
- Conduct a thorough risk analysis annually and after significant system changes
- Use only HIPAA-eligible cloud services and sign BAAs before deploying PHI workloads
- Encrypt all PHI at rest and in transit with no exceptions
- Implement the minimum necessary standard: grant access only to the PHI needed for each role
- Maintain audit logs of all PHI access for a minimum of 6 years
- Train all workforce members on HIPAA policies at onboarding and annually
- Test contingency plans (backup restore, DR failover, emergency access) at least annually
- Track all Business Associate Agreements in a central registry with review dates
- Document every addressable specification decision (implement, alternative, or not applicable with rationale)
- Prepare breach notification templates and procedures before an incident occurs
compliance/frameworks/iso27001-compliance/SKILL.md
+406 -54
View File
@@ -9,77 +9,429 @@ metadata:
# ISO 27001 Compliance
Implement ISO 27001 Information Security Management System.
Implement an Information Security Management System (ISMS) aligned with ISO/IEC 27001:2022.
## ISMS Framework
## When to Use
- Establishing an ISMS for the first time in an organization
- Preparing for ISO 27001 certification audit
- Conducting risk assessments and developing risk treatment plans
- Creating the Statement of Applicability (SoA)
- Transitioning from ISO 27001:2013 to the 2022 revision
- Meeting customer or regulatory requirements for ISO 27001 certification
## ISMS Plan-Do-Check-Act Cycle
```yaml
plan_do_check_act:
pdca_cycle:
plan:
- Define scope
- Risk assessment
- Risk treatment plan
- Statement of Applicability
- Define ISMS scope and boundaries
- Establish information security policy
- Conduct risk assessment
- Develop risk treatment plan
- Produce Statement of Applicability
- Obtain management approval and commitment
- Define security objectives and metrics
do:
- Implement controls
- Security awareness
- Document procedures
- Implement selected Annex A controls
- Deploy technical security controls
- Conduct security awareness training
- Document all procedures and processes
- Implement incident management process
- Establish supplier security management
check:
- Internal audits
- Management review
- Performance measurement
- Conduct internal audits (at least annual)
- Perform management review meetings
- Monitor and measure control effectiveness
- Review incident trends and near misses
- Assess compliance with legal requirements
- Evaluate security metrics against objectives
act:
- Corrective actions
- Continual improvement
- Address nonconformities with corrective actions
- Implement continual improvement initiatives
- Update risk assessment based on changes
- Refine controls based on audit findings
- Communicate improvements to stakeholders
```
## Annex A Controls
## ISMS Scope Definition
```yaml
control_domains:
A.5: Information security policies
A.6: Organization of information security
A.7: Human resource security
A.8: Asset management
A.9: Access control
A.10: Cryptography
A.11: Physical security
A.12: Operations security
A.13: Communications security
A.14: System acquisition/development
A.15: Supplier relationships
A.16: Incident management
A.17: Business continuity
A.18: Compliance
isms_scope:
template:
organization: "Company Name, Ltd."
scope_statement: |
The ISMS covers the design, development, operation, and support of
the Company's cloud-based SaaS platform, including all supporting
infrastructure, personnel, and processes at the following locations.
included:
locations:
- "Primary office: 123 Main Street, City, Country"
- "AWS us-east-1 and eu-west-1 regions"
- "Remote workers accessing corporate systems"
business_processes:
- "Software development and deployment"
- "Cloud infrastructure management"
- "Customer data processing and storage"
- "Customer support operations"
- "Corporate IT and internal systems"
information_assets:
- "Customer data (PII, business data)"
- "Source code and intellectual property"
- "Employee personal data"
- "Financial records"
- "Security configurations and credentials"
technology:
- "AWS cloud infrastructure"
- "SaaS application stack"
- "Corporate IT systems (Google Workspace, Okta, Jira)"
- "Development tools (GitHub, CI/CD pipelines)"
excluded:
- "Physical data center operations (inherited from AWS)"
- "Third-party SaaS platforms beyond integration points"
exclusion_justification: "Physical data center controls are inherited from AWS, which maintains its own ISO 27001 certification."
interfaces:
- "Customer API endpoints"
- "Third-party integrations (payment processor, email provider)"
- "AWS management plane"
```
## Risk Assessment
## Risk Assessment Process
```yaml
risk_assessment:
identify:
- Asset inventory
- Threat identification
- Vulnerability assessment
analyze:
- Likelihood rating
- Impact rating
- Risk calculation
evaluate:
- Risk acceptance criteria
- Prioritization
- Treatment options
methodology:
approach: "Asset-based risk assessment"
risk_formula: "Risk = Likelihood x Impact"
scale: "1-5 for both likelihood and impact (total 1-25)"
likelihood_scale:
1: "Rare - less than once per 5 years"
2: "Unlikely - once per 2-5 years"
3: "Possible - once per 1-2 years"
4: "Likely - multiple times per year"
5: "Almost Certain - monthly or more frequent"
impact_scale:
1: "Negligible - minimal operational impact, no data loss"
2: "Minor - limited impact, small data exposure, <$10K cost"
3: "Moderate - significant impact, data breach <1K records, <$100K cost"
4: "Major - severe impact, large data breach, <$1M cost, regulatory action"
5: "Critical - catastrophic, massive breach, >$1M cost, business viability at risk"
risk_matrix:
# Impact: 1 2 3 4 5
likelihood_5: [5, 10, 15, 20, 25]
likelihood_4: [4, 8, 12, 16, 20]
likelihood_3: [3, 6, 9, 12, 15]
likelihood_2: [2, 4, 6, 8, 10]
likelihood_1: [1, 2, 3, 4, 5]
risk_appetite:
accept: "Score 1-4 (low risk, accept with monitoring)"
mitigate: "Score 5-14 (medium risk, implement controls to reduce)"
escalate: "Score 15-25 (high/critical risk, immediate action required)"
treatment_options:
mitigate: "Implement controls to reduce likelihood or impact"
transfer: "Insurance or contractual transfer to third party"
avoid: "Eliminate the risk by removing the activity or asset"
accept: "Accept with documented management approval"
example_risk_register:
- id: "RISK-001"
asset: "Customer database"
threat: "SQL injection attack"
vulnerability: "Insufficient input validation"
likelihood: 3
impact: 4
inherent_risk: 12
treatment: "mitigate"
controls: ["A.8.28 Secure coding", "A.8.8 Vulnerability management"]
residual_likelihood: 1
residual_impact: 4
residual_risk: 4
risk_owner: "CTO"
- id: "RISK-002"
asset: "Source code repository"
threat: "Insider theft of intellectual property"
vulnerability: "Excessive access permissions"
likelihood: 2
impact: 5
inherent_risk: 10
treatment: "mitigate"
controls: ["A.5.15 Access control", "A.8.3 Information access restriction"]
residual_likelihood: 1
residual_impact: 5
residual_risk: 5
risk_owner: "VP Engineering"
- id: "RISK-003"
asset: "Cloud infrastructure"
threat: "Cloud provider outage"
vulnerability: "Single-region deployment"
likelihood: 3
impact: 3
inherent_risk: 9
treatment: "mitigate"
controls: ["A.5.30 ICT readiness for business continuity", "A.8.14 Redundancy"]
residual_likelihood: 3
residual_impact: 2
residual_risk: 6
risk_owner: "Head of Infrastructure"
```
## Statement of Applicability (SoA)
```yaml
# ISO 27001:2022 Annex A Controls - Statement of Applicability
soa_template:
organizational_controls_5:
"A.5.1":
control: "Policies for information security"
applicable: true
justification: "Required to establish security governance"
implementation: "Information security policy approved by CEO, reviewed annually"
"A.5.2":
control: "Information security roles and responsibilities"
applicable: true
justification: "Required for accountability"
implementation: "RACI matrix for security responsibilities, CISO appointed"
"A.5.7":
control: "Threat intelligence"
applicable: true
justification: "Required for proactive threat management"
implementation: "Subscribe to threat feeds, CVE monitoring, vendor advisories"
"A.5.15":
control: "Access control"
applicable: true
justification: "Required for data protection"
implementation: "RBAC via Okta, least-privilege IAM policies, quarterly access reviews"
"A.5.23":
control: "Information security for use of cloud services"
applicable: true
justification: "Primary infrastructure is cloud-based"
implementation: "AWS security baseline, CSP shared responsibility documented"
"A.5.29":
control: "Information security during disruption"
applicable: true
justification: "Business continuity requirement"
implementation: "BCP/DR plans tested annually, multi-AZ deployment"
"A.5.30":
control: "ICT readiness for business continuity"
applicable: true
justification: "Ensure technology supports continuity"
implementation: "DR runbooks, RTO/RPO defined, failover tested quarterly"
people_controls_6:
"A.6.1":
control: "Screening"
applicable: true
implementation: "Background checks for all employees before hiring"
"A.6.3":
control: "Information security awareness, education and training"
applicable: true
implementation: "Annual security training, phishing simulations quarterly"
"A.6.5":
control: "Responsibilities after termination or change of employment"
applicable: true
implementation: "Offboarding checklist, access revoked within 24 hours"
physical_controls_7:
"A.7.1":
control: "Physical security perimeters"
applicable: false
exclusion_justification: "No company-operated data centers, inherited from AWS"
technology_controls_8:
"A.8.1":
control: "User endpoint devices"
applicable: true
implementation: "MDM enrollment, disk encryption, screen lock policy"
"A.8.5":
control: "Secure authentication"
applicable: true
implementation: "MFA required for all systems, SSO via Okta"
"A.8.8":
control: "Management of technical vulnerabilities"
applicable: true
implementation: "Weekly vulnerability scans, 30-day patch SLA for critical"
"A.8.9":
control: "Configuration management"
applicable: true
implementation: "Infrastructure as code, AWS Config rules, baseline hardening"
"A.8.15":
control: "Logging"
applicable: true
implementation: "Centralized logging via CloudWatch + SIEM, 12-month retention"
"A.8.16":
control: "Monitoring activities"
applicable: true
implementation: "SIEM alerting, 24/7 on-call rotation, anomaly detection"
"A.8.24":
control: "Use of cryptography"
applicable: true
implementation: "TLS 1.2+, AES-256 at rest, KMS key management"
"A.8.25":
control: "Secure development lifecycle"
applicable: true
implementation: "SAST/DAST in CI, code review required, dependency scanning"
"A.8.28":
control: "Secure coding"
applicable: true
implementation: "OWASP guidelines, security code review, automated linting"
```
## Internal Audit Program
```yaml
internal_audit:
schedule:
frequency: "Annual full cycle, quarterly focused audits"
cycle: "All ISMS clauses and applicable Annex A controls audited over 12 months"
audit_plan_template:
audit_id: "IA-2025-Q1"
scope: "Clauses 4-10, Annex A controls A.5.1-A.5.15"
auditor: "Internal auditor (independent of audited area)"
audit_dates: "2025-03-10 to 2025-03-14"
areas:
- area: "Access Control (A.5.15)"
auditee: "IT Security Team"
evidence_requested:
- "Access review records from last quarter"
- "Joiner/mover/leaver process records"
- "Privileged access management logs"
- area: "Risk Management (Clause 6.1)"
auditee: "Risk Management Team"
evidence_requested:
- "Current risk register"
- "Risk assessment methodology document"
- "Management risk review meeting minutes"
finding_categories:
major_nonconformity: "Requirement not met, significant risk to ISMS effectiveness"
minor_nonconformity: "Requirement partially met, limited risk"
observation: "Area for improvement, no requirement breach"
positive_finding: "Notably effective implementation"
corrective_action:
major: "Root cause analysis within 10 days, corrective action within 30 days"
minor: "Corrective action within 60 days"
observation: "Address in next ISMS review cycle"
verification: "Auditor verifies corrective action effectiveness"
```
## Management Review Meeting
```yaml
management_review:
frequency: "At least annually, recommended quarterly"
attendees:
required:
- "CEO or Managing Director"
- "CISO or Information Security Manager"
- "Department heads"
optional:
- "Internal auditor"
- "Risk manager"
- "External consultant"
mandatory_inputs:
- "Status of actions from previous management reviews"
- "Changes in external and internal issues relevant to the ISMS"
- "Information security performance (metrics and KPIs)"
- "Audit results (internal and external)"
- "Incident trends and nonconformities"
- "Risk assessment results and risk treatment plan status"
- "Interested party feedback"
- "Opportunities for continual improvement"
mandatory_outputs:
- "Decisions on continual improvement opportunities"
- "Decisions on changes needed to the ISMS"
- "Resource allocation decisions"
- "Updated risk acceptance decisions"
kpis_to_report:
- "Number and severity of security incidents"
- "Vulnerability remediation SLA compliance"
- "Security awareness training completion rate"
- "Access review completion rate"
- "Audit finding closure rate"
- "Risk treatment plan progress"
- "Patch compliance percentage"
```
## ISO 27001 Certification Checklist
```yaml
certification_checklist:
stage_1_audit_preparation:
- [ ] ISMS scope documented and approved
- [ ] Information security policy published
- [ ] Risk assessment methodology defined
- [ ] Risk assessment completed with risk register
- [ ] Risk treatment plan developed
- [ ] Statement of Applicability completed
- [ ] ISMS objectives defined with measurable targets
- [ ] Internal audit program established
- [ ] At least one full internal audit completed
- [ ] Management review conducted with minutes documented
- [ ] Document control process in place
stage_2_audit_preparation:
- [ ] All Annex A controls implemented per SoA
- [ ] Evidence of control operation for 3+ months
- [ ] Corrective actions from internal audit tracked and closed
- [ ] Security awareness training delivered and recorded
- [ ] Incident management process operational with records
- [ ] Supplier security assessments performed
- [ ] Business continuity plan tested
- [ ] All mandatory documented information available
- [ ] Employees aware of security policy and their responsibilities
surveillance_audit_readiness:
- [ ] All corrective actions from certification audit closed
- [ ] Continuous internal audit schedule maintained
- [ ] Management reviews conducted per schedule
- [ ] Risk register updated with new threats and changes
- [ ] Metrics demonstrate ISMS effectiveness
- [ ] Changes to ISMS scope documented
```
## Best Practices
- Management commitment
- Risk-based approach
- Document everything
- Regular internal audits
- Continuous improvement
- Secure visible management commitment with a signed information security policy
- Define ISMS scope carefully; too broad makes certification expensive, too narrow reduces value
- Use an asset-based risk assessment approach to ensure comprehensive coverage
- Maintain the Statement of Applicability as a living document aligned with the risk register
- Conduct internal audits with auditors independent of the area being audited
- Hold management review meetings quarterly rather than only annually
- Integrate ISO 27001 controls into daily operations rather than treating them as a separate compliance exercise
- Use metrics and KPIs to demonstrate ISMS effectiveness to auditors and management
- Plan for the 3-year certification cycle: certification audit, then two surveillance audits
- Start collecting evidence of control operation at least 3 months before the Stage 2 audit
compliance/frameworks/pci-dss-compliance/SKILL.md
+396 -50
View File
@@ -9,69 +9,415 @@ metadata:
# PCI DSS Compliance
Implement PCI DSS requirements for payment card security.
Implement PCI DSS v4.0 requirements for protecting cardholder data across the Cardholder Data Environment (CDE), including network segmentation, encryption, access controls, and ongoing testing.
## Requirements
## When to Use
- Processing, storing, or transmitting payment card data
- Scoping the Cardholder Data Environment for PCI assessment
- Selecting the appropriate Self-Assessment Questionnaire (SAQ)
- Implementing network segmentation to reduce CDE scope
- Preparing for QSA assessment or ASV scanning
## SAQ Types and Applicability
```yaml
saq_types:
SAQ_A:
description: "Card-not-present merchants using fully outsourced payment"
applies_when:
- All payment processing fully outsourced to PCI-compliant third party
- No electronic storage, processing, or transmission of cardholder data
- Only payment page redirects or iframes from compliant provider
requirements: ~22 questions
SAQ_A_EP:
description: "E-commerce merchants with website that affects payment security"
applies_when:
- E-commerce channel only
- Website controls redirect to or loads payment page from third party
- No direct processing but website could affect transaction security
requirements: ~191 questions
SAQ_B:
description: "Merchants with only imprint machines or standalone terminals"
applies_when:
- Only standalone POS terminals (dial-out or IP connected)
- No electronic cardholder data storage
- No e-commerce channel
requirements: ~41 questions
SAQ_C:
description: "Merchants with payment application systems connected to internet"
applies_when:
- Payment application connected to internet
- No electronic cardholder data storage
- No e-commerce channel
requirements: ~160 questions
SAQ_D:
description: "All other merchants and all service providers"
applies_when:
- Stores cardholder data electronically
- Does not fit any other SAQ type
- Service providers eligible for SAQ D
requirements: "Full set of PCI DSS requirements"
scope_reduction_strategies:
- Use tokenization to replace PAN with non-sensitive tokens
- Use P2PE (Point-to-Point Encryption) validated solutions
- Outsource payment processing to reduce your CDE footprint
- Implement network segmentation to isolate CDE
```
## PCI DSS v4.0 Requirements Overview
```yaml
requirements:
1_firewall:
- Network segmentation
- Firewall configuration
- CDE isolation
3_protect_data:
- Mask PAN display
- Encrypt stored data
- Key management
6_secure_systems:
- Patch management
- Secure development
- Change control
8_access_control:
- Unique IDs
- MFA for remote access
- Password policies
10_logging:
- Audit trail
- Time synchronization
- Log retention (1 year)
11_testing:
- Vulnerability scans
- Penetration testing
- IDS/IPS monitoring
req_1_network_security:
"1.1": "Network security controls defined and maintained"
"1.2": "Network security controls configured and maintained"
"1.3": "Network access to and from CDE is restricted"
"1.4": "Network connections between trusted and untrusted networks controlled"
"1.5": "Risks to CDE from devices connecting to untrusted networks mitigated"
req_2_secure_configuration:
"2.1": "Secure configuration standards defined and applied"
"2.2": "System components configured and managed securely"
req_3_protect_stored_data:
"3.1": "Processes for protecting stored account data defined"
"3.2": "Storage of account data is minimized"
"3.3": "Sensitive authentication data not stored after authorization"
"3.4": "PAN masked when displayed (first 6, last 4 maximum)"
"3.5": "PAN secured wherever stored"
"3.6": "Cryptographic keys managed securely"
"3.7": "Key management procedures documented"
req_4_transmission_encryption:
"4.1": "Strong cryptography protects cardholder data during transmission"
"4.2": "PAN protected when sent via end-user messaging"
req_5_malware_protection:
"5.1": "Processes to protect against malware defined"
"5.2": "Malware prevented or detected and addressed"
"5.3": "Anti-malware mechanisms active and maintained"
"5.4": "Anti-phishing mechanisms protect against phishing"
req_6_secure_development:
"6.1": "Secure development processes defined"
"6.2": "Bespoke and custom software developed securely"
"6.3": "Security vulnerabilities identified and addressed"
"6.4": "Public-facing web applications protected against attacks"
"6.5": "Changes to all system components managed securely"
req_7_access_restriction:
"7.1": "Access to system components and data restricted by business need"
"7.2": "Access appropriately defined and assigned"
"7.3": "Access to system components and data managed via access control"
req_8_user_identification:
"8.1": "Processes for user identification defined"
"8.2": "User identification and accounts managed"
"8.3": "Strong authentication established"
"8.4": "MFA implemented for all access into CDE"
"8.5": "MFA systems configured to prevent misuse"
"8.6": "System and application accounts managed"
req_9_physical_access:
"9.1": "Physical access controls defined"
"9.2": "Physical access to CDE managed"
"9.3": "Physical access for personnel and visitors authorized"
"9.4": "Media with cardholder data managed securely"
"9.5": "POI devices protected from tampering"
req_10_logging:
"10.1": "Audit logging processes defined"
"10.2": "Audit logs record required events"
"10.3": "Audit logs protected from destruction and modification"
"10.4": "Audit logs reviewed for anomalies"
"10.5": "Audit log history retained"
"10.6": "Time synchronization mechanisms configured"
"10.7": "Audit logs retained for at least 12 months (3 months immediately available)"
req_11_testing:
"11.1": "Security testing processes defined"
"11.2": "Wireless access points managed"
"11.3": "Vulnerabilities identified and addressed"
"11.4": "External and internal penetration testing performed"
"11.5": "Network intrusions and changes detected and responded to"
"11.6": "Unauthorized changes to payment pages detected"
req_12_policies:
"12.1": "Information security policy established"
"12.2": "Acceptable use policies defined"
"12.3": "Risks to CDE formally identified and managed"
"12.4": "PCI DSS compliance managed"
"12.5": "PCI DSS scope documented and validated"
"12.6": "Security awareness program"
"12.8": "Third-party service providers managed"
"12.9": "TPSPs acknowledge responsibility for cardholder data"
"12.10": "Security incidents responded to immediately"
```
## Network Segmentation
## Network Segmentation Architecture
```
Internet --> DMZ --> Firewall --> CDE
|
Non-CDE <-- Firewall --
┌──────────────────────────────────────┐
│ INTERNET │
└──────────────┬───────────────────────┘
│
┌──────────────▼───────────────────────┐
│ DMZ (Public Subnet) │
│ WAF → Load Balancer → Web Servers │
└──────────────┬───────────────────────┘
│ Firewall (Req 1.3)
┌──────────────▼───────────────────────┐
│ CDE (Cardholder Data Environment) │
│ ┌─────────┐ ┌──────────┐ │
│ │ Payment │ │ Card DB │ │
│ │ App │ │(encrypted)│ │
│ └─────────┘ └──────────┘ │
│ ┌─────────┐ ┌──────────┐ │
│ │Token Svc│ │ HSM/KMS │ │
│ └─────────┘ └──────────┘ │
└──────────────┬───────────────────────┘
│ Firewall (Req 1.3)
┌──────────────▼───────────────────────┐
│ Non-CDE (Corporate Network) │
│ App servers, internal tools │
│ (no cardholder data) │
└──────────────────────────────────────┘
```
## Data Protection
```bash
# AWS Security Group for CDE isolation
aws ec2 create-security-group \
--group-name cde-app-sg \
--description "CDE Application Security Group" \
--vpc-id vpc-CDE
# Allow only HTTPS from WAF/ALB
aws ec2 authorize-security-group-ingress \
--group-id sg-CDE-APP \
--protocol tcp --port 443 \
--source-group sg-ALB
# CDE database - only accessible from CDE app servers
aws ec2 create-security-group \
--group-name cde-db-sg \
--description "CDE Database Security Group" \
--vpc-id vpc-CDE
aws ec2 authorize-security-group-ingress \
--group-id sg-CDE-DB \
--protocol tcp --port 5432 \
--source-group sg-CDE-APP
# Deny all other inbound by default (security groups are deny-all by default in AWS)
# Document all rules for Req 1.2 - firewall/security group documentation
```
## Encryption and Tokenization
```yaml
encryption:
at_rest: AES-256
in_transit: TLS 1.2+
key_storage: HSM or dedicated key vault
tokenization:
- Replace PAN with token
- Store mapping securely
- Reduce CDE scope
encryption_requirements:
stored_data_req_3:
pan_encryption:
algorithm: AES-256
mode: GCM (preferred) or CBC with HMAC
key_storage: HSM or dedicated key management service
never_store:
- Full track data (magnetic stripe)
- CVV/CVC/CAV2
- PIN / PIN block
pan_display_masking:
rule: "Show maximum first 6 and last 4 digits"
examples:
masked: "4111 11** **** 1111"
acceptable_for_business: "First 6 and last 4"
implementation: "Apply masking at application layer before rendering"
key_management_req_3_6:
- Generate keys using approved random number generator
- Protect keys with key-encrypting keys (KEKs)
- Store key components separately (split knowledge, dual control)
- Rotate keys at least annually (or per crypto period)
- Retire and replace keys when compromised
- Document key custodian responsibilities
transmission_req_4:
protocols:
required: "TLS 1.2 or higher"
prohibited: "SSL, TLS 1.0, TLS 1.1"
cipher_suites:
preferred:
- TLS_AES_256_GCM_SHA384
- TLS_CHACHA20_POLY1305_SHA256
minimum: "128-bit key strength"
certificate_management:
- Use certificates from trusted CAs
- Verify hostname and certificate validity
- Monitor certificate expiration
tokenization_strategy:
description: "Replace PAN with non-reversible token to reduce CDE scope"
implementation:
- Use format-preserving tokens (same length/format as PAN)
- Token vault in isolated CDE segment
- Token-to-PAN mapping encrypted and access-controlled
- De-tokenization requires authenticated API call
- Log all de-tokenization requests
scope_benefit: "Systems using only tokens are out of PCI scope"
```
## Vulnerability Management and Testing
```bash
# Req 11.3 - Internal vulnerability scanning (quarterly minimum)
# Using OpenVAS or Nessus
openvas-cli --scan-target 10.10.0.0/24 --scan-name "CDE-Quarterly-Scan" \
--profile "PCI DSS" --output pci-scan-$(date +%Y%m%d).xml
# Req 11.3 - External ASV scanning (quarterly, must pass)
# Schedule with Approved Scanning Vendor (Qualys, Tenable, etc.)
# ASV scan must show no vulnerabilities with CVSS >= 4.0
# Req 6.3 - Patch management
# Check for critical patches on CDE systems
yum check-update --security # RHEL/CentOS
apt list --upgradable 2>/dev/null | grep -i security # Debian/Ubuntu
# Req 11.4 - Penetration testing (annual for external, internal, and segmentation)
# Must be performed by qualified internal resource or third party
# Test both network layer and application layer
# Segmentation testing: verify CDE is isolated from non-CDE networks
# Req 11.5 - File integrity monitoring
# Using AIDE (Advanced Intrusion Detection Environment)
aide --init # Initialize baseline
aide --check # Compare against baseline
# OSSEC FIM configuration for CDE systems
# /var/ossec/etc/ossec.conf
# <syscheck>
# <frequency>3600</frequency>
# <directories check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
# <directories check_all="yes">/opt/payment-app</directories>
# </syscheck>
```
## Logging and Monitoring (Req 10)
```yaml
required_audit_events:
"10.2.1": "All individual user accesses to cardholder data"
"10.2.2": "All actions taken by any individual with root or admin privileges"
"10.2.3": "Access to all audit trails"
"10.2.4": "Invalid logical access attempts"
"10.2.5": "Changes to identification and authentication credentials"
"10.2.6": "Initialization, stopping, or pausing of audit logs"
"10.2.7": "Creation and deletion of system-level objects"
log_entry_requirements:
"10.3.1": "User identification"
"10.3.2": "Type of event"
"10.3.3": "Date and time"
"10.3.4": "Success or failure indication"
"10.3.5": "Origination of event"
"10.3.6": "Identity or name of affected data/resource"
retention:
minimum: "12 months total"
immediately_available: "At least 3 months"
archive: "Remaining months can be in archive storage"
time_synchronization:
"10.6.1": "Synchronize clocks using NTP"
"10.6.2": "Time data protected from unauthorized access"
"10.6.3": "Time settings received from industry-accepted sources"
ntp_config: |
# /etc/ntp.conf or chrony.conf for CDE systems
server 0.pool.ntp.org iburst
server 1.pool.ntp.org iburst
driftfile /var/lib/ntp/drift
restrict default nomodify notrap nopeer noquery
restrict 127.0.0.1
```
## PCI DSS Compliance Checklist
```yaml
pci_dss_checklist:
scoping:
- [ ] CDE boundaries identified and documented
- [ ] All in-scope systems inventoried
- [ ] Network segmentation validated
- [ ] Data flow diagrams current and accurate
- [ ] SAQ type determined (if applicable)
- [ ] Third-party service providers identified
network_security:
- [ ] Firewalls/security groups restrict CDE access
- [ ] Default deny rules on all CDE boundaries
- [ ] Wireless networks segmented from CDE
- [ ] Remote access uses MFA
- [ ] All firewall rules documented with business justification
- [ ] Rules reviewed semi-annually
data_protection:
- [ ] PAN masked when displayed (first 6, last 4 max)
- [ ] Stored PAN encrypted with AES-256 or equivalent
- [ ] Sensitive auth data not stored after authorization
- [ ] Encryption keys managed per Req 3.6/3.7
- [ ] TLS 1.2+ for all cardholder data transmission
- [ ] Tokenization implemented where feasible
access_control:
- [ ] Access restricted on need-to-know basis
- [ ] Unique IDs for all users
- [ ] MFA for all access into CDE
- [ ] MFA for all remote/non-console admin access
- [ ] Default/vendor passwords changed
- [ ] Shared/group accounts not used (or tightly controlled)
- [ ] Access reviewed at least every 6 months
monitoring:
- [ ] Audit logs capture all required events (Req 10.2)
- [ ] Log entries include all required fields (Req 10.3)
- [ ] Logs protected from modification
- [ ] Logs retained 12 months (3 months immediately available)
- [ ] Time synchronization configured (NTP)
- [ ] Daily log review process or automated alerting
- [ ] File integrity monitoring on critical files
testing:
- [ ] Internal vulnerability scans quarterly
- [ ] External ASV scans quarterly (passing)
- [ ] Internal penetration test annually
- [ ] External penetration test annually
- [ ] Segmentation test annually (or after changes)
- [ ] Web application assessment annually (or WAF deployed)
- [ ] IDS/IPS monitoring all CDE network traffic
policies:
- [ ] Information security policy reviewed annually
- [ ] Security awareness training for all personnel
- [ ] Incident response plan documented and tested
- [ ] Third-party service provider compliance confirmed
- [ ] Risk assessment performed annually
```
## Best Practices
- Minimize CDE scope
- Use tokenization
- Quarterly vulnerability scans
- Annual penetration tests
- ASV scan certification
- Minimize CDE scope aggressively using tokenization, P2PE, and outsourced payment processing
- Use network segmentation to isolate the CDE and reduce the number of in-scope systems
- Never store sensitive authentication data (CVV, track data, PIN) after authorization
- Implement MFA for all access into the CDE, not just remote access (v4.0 requirement)
- Automate vulnerability scanning and patch management to maintain continuous compliance
- Deploy file integrity monitoring on all CDE systems to detect unauthorized changes
- Synchronize clocks across all CDE systems using NTP for accurate log correlation
- Conduct internal and external penetration tests annually and after significant changes
- Review all firewall and security group rules semi-annually with documented business justification
- Maintain a current data flow diagram showing all cardholder data transmission and storage points
compliance/frameworks/soc2-compliance/SKILL.md
+356 -52
View File
@@ -9,74 +9,378 @@ metadata:
# SOC 2 Compliance
Implement SOC 2 Trust Services Criteria for certification.
Implement SOC 2 Trust Services Criteria controls, evidence collection, and continuous compliance monitoring for Type I and Type II audits.
## Trust Services Criteria
## When to Use
- Preparing for a SOC 2 Type I or Type II audit
- Mapping existing controls to Trust Services Criteria
- Automating evidence collection for auditor requests
- Building continuous compliance monitoring into CI/CD
- Onboarding new services and ensuring SOC 2 control coverage
## Trust Services Criteria Detailed Checklist
```yaml
criteria:
security:
- Access controls
- Change management
- Risk assessment
- Incident response
availability:
- System monitoring
- Disaster recovery
- Capacity planning
- SLA management
processing_integrity:
- Input validation
- Processing completeness
- Output accuracy
confidentiality:
- Data classification
- Encryption
- Access restrictions
privacy:
- Data collection notice
- Consent management
- Data retention
security_common_criteria:
CC1_control_environment:
CC1.1: "Management demonstrates commitment to integrity and ethical values"
CC1.2: "Board exercises oversight of internal controls"
CC1.3: "Management establishes structure, authority, and responsibility"
CC1.4: "Commitment to competence - hire and retain qualified personnel"
CC1.5: "Individuals are held accountable for internal control responsibilities"
evidence:
- Code of conduct document
- Organizational chart
- Job descriptions with security responsibilities
- Board meeting minutes discussing security
- Background check policy and records
CC2_communication:
CC2.1: "Entity obtains or generates relevant quality information"
CC2.2: "Entity internally communicates information including objectives and responsibilities"
CC2.3: "Entity communicates with external parties"
evidence:
- Security awareness training records
- Internal security newsletters or updates
- Customer-facing security documentation
- Status page and incident communication records
CC3_risk_assessment:
CC3.1: "Entity specifies objectives clearly to identify and assess risks"
CC3.2: "Entity identifies risks to achievement of objectives"
CC3.3: "Entity considers potential for fraud"
CC3.4: "Entity identifies and assesses significant changes"
evidence:
- Annual risk assessment report
- Risk register with ratings and treatment plans
- Fraud risk assessment documentation
- Change management records
CC4_monitoring:
CC4.1: "Entity selects, develops, and performs ongoing/separate evaluations"
CC4.2: "Entity evaluates and communicates internal control deficiencies"
evidence:
- Continuous monitoring dashboard screenshots
- Internal audit reports
- Vulnerability scan results
- Penetration test reports
CC5_control_activities:
CC5.1: "Entity selects and develops control activities to mitigate risks"
CC5.2: "Entity selects and develops technology-based controls"
CC5.3: "Entity deploys control activities through policies and procedures"
evidence:
- Information security policy
- Access control procedures
- Change management procedures
- Encryption standards documentation
CC6_logical_access:
CC6.1: "Logical access security over protected information assets"
CC6.2: "Prior to access, users are registered and authorized"
CC6.3: "Access to data, software, functions, and other IT resources is authorized and modified"
CC6.6: "Logical access security measures against threats from outside system boundaries"
CC6.7: "Transmission of data between parties is protected"
CC6.8: "Controls to prevent or detect unauthorized or malicious software"
evidence:
- IAM credential report
- MFA enforcement configuration
- Access review completion records
- Firewall and WAF configurations
- TLS/encryption configurations
- Endpoint protection deployment records
CC7_system_operations:
CC7.1: "Detect anomalies and potential security incidents"
CC7.2: "Monitor system components for anomalies"
CC7.3: "Evaluate detected events and determine incidents"
CC7.4: "Respond to identified security incidents"
CC7.5: "Identify and remediate security incidents"
evidence:
- SIEM alert rules and dashboards
- Monitoring configuration (CloudWatch, Datadog, etc.)
- Incident response plan
- Incident tickets and post-mortems
CC8_change_management:
CC8.1: "Entity authorizes, designs, develops, configures, documents, tests, approves, and implements changes"
evidence:
- Change management policy
- Pull request approval requirements
- CI/CD pipeline configurations
- Deployment records with approvals
CC9_risk_mitigation:
CC9.1: "Entity identifies, selects, and develops risk mitigation activities"
CC9.2: "Entity assesses and manages risks associated with vendors"
evidence:
- Risk treatment plans
- Vendor assessment records
- Business associate agreements
- Insurance certificates
availability_criteria:
A1.1: "System processing capacity and availability are maintained"
A1.2: "Environmental protections and recovery measures"
A1.3: "Recovery plan procedures to support system availability"
evidence:
- Uptime SLA documentation
- Capacity monitoring dashboards
- Disaster recovery plan
- DR test results
- Backup verification records
processing_integrity_criteria:
PI1.1: "Entity obtains or generates, uses, and communicates quality information"
evidence:
- Input validation procedures
- Data processing accuracy checks
- Error handling and retry logic documentation
- Output reconciliation records
confidentiality_criteria:
C1.1: "Entity identifies and maintains confidential information"
C1.2: "Entity disposes of confidential information"
evidence:
- Data classification policy
- Encryption configurations
- Data retention and destruction policies
- Secure disposal records
privacy_criteria:
P1-P8: "Privacy notice, choice, collection, use, disclosure, access, quality, monitoring"
evidence:
- Privacy policy (published)
- Consent management records
- Data processing inventory
- DSAR handling procedures
```
## Key Controls
## Tool Mappings for Control Evidence
```yaml
controls:
control_to_tool_mapping:
CC6.1_logical_access:
- MFA enforcement
- Role-based access
- Access reviews
aws:
- IAM credential report (aws iam generate-credential-report)
- IAM Access Analyzer findings
- AWS SSO configuration
- GuardDuty findings
azure:
- Azure AD sign-in logs
- Conditional Access policies
- PIM role assignments
github:
- Organization member list and roles
- Repository access permissions
- Branch protection rules
okta:
- User status report
- MFA enrollment report
- Application assignment report
CC7.2_monitoring:
- Log aggregation
- Alert thresholds
- Incident tracking
tools:
- CloudWatch / Azure Monitor / Cloud Monitoring dashboards
- Datadog / New Relic / Grafana alert configurations
- SIEM (Splunk, Elastic, Sentinel) saved searches
- PagerDuty / OpsGenie escalation policies
evidence_format:
- Dashboard screenshots with date stamps
- Alert rule configuration exports
- Incident response records from ticketing system
CC8.1_change_management:
- Change requests
- Approval workflows
- Testing requirements
tools:
- GitHub/GitLab PR merge requirements
- CI/CD pipeline configurations (GitHub Actions, Jenkins)
- Terraform plan outputs
- Deployment logs
evidence_format:
- PR with approvals and CI checks
- Deployment audit trail
- Change advisory board meeting notes (if applicable)
```
## Evidence Collection
## Evidence Collection Automation
```bash
# Access review export
aws iam generate-credential-report
aws iam get-credential-report
#!/usr/bin/env bash
# collect-soc2-evidence.sh - Automated SOC 2 evidence collection
# Run monthly or before audit requests
# Audit logs
aws cloudtrail lookup-events --start-time $(date -d '30 days ago' --iso)
EVIDENCE_DIR="./soc2-evidence/$(date +%Y-%m)"
mkdir -p "$EVIDENCE_DIR"
echo "=== CC6.1 - Logical Access Evidence ==="
# AWS IAM credential report
aws iam generate-credential-report
sleep 10
aws iam get-credential-report --output text --query Content | \
base64 -d > "$EVIDENCE_DIR/aws-iam-credential-report.csv"
# AWS IAM Access Analyzer findings
aws accessanalyzer list-findings \
--analyzer-arn "arn:aws:access-analyzer:us-east-1:123456789012:analyzer/org-analyzer" \
--filter '{"status": {"eq": ["ACTIVE"]}}' \
> "$EVIDENCE_DIR/access-analyzer-findings.json"
# MFA enforcement status
aws iam list-users --query 'Users[*].UserName' --output text | \
tr '\t' '\n' | while read -r user; do
mfa=$(aws iam list-mfa-devices --user-name "$user" --query 'MFADevices[0].SerialNumber' --output text)
echo "$user,$mfa"
done > "$EVIDENCE_DIR/mfa-status.csv"
# GitHub organization members and roles
gh api orgs/YOUR_ORG/members --paginate --jq '.[] | [.login, .role_name // "member"] | @csv' \
> "$EVIDENCE_DIR/github-org-members.csv"
# GitHub branch protection rules
for repo in $(gh repo list YOUR_ORG --json name -q '.[].name'); do
gh api repos/YOUR_ORG/$repo/branches/main/protection \
> "$EVIDENCE_DIR/branch-protection-$repo.json" 2>/dev/null
done
echo "=== CC7.2 - Monitoring Evidence ==="
# CloudTrail status
aws cloudtrail get-trail-status --name org-audit-trail \
> "$EVIDENCE_DIR/cloudtrail-status.json"
# Active CloudWatch alarms
aws cloudwatch describe-alarms --state-value ALARM \
> "$EVIDENCE_DIR/active-alarms.json"
# GuardDuty findings summary
aws guardduty list-findings --detector-id DETECTOR_ID \
--finding-criteria '{"criterion":{"severity":{"gte":4}}}' \
> "$EVIDENCE_DIR/guardduty-findings.json"
echo "=== CC8.1 - Change Management Evidence ==="
# Recent deployments (GitHub Actions)
gh run list --repo YOUR_ORG/YOUR_REPO --limit 50 --json conclusion,createdAt,displayTitle,headBranch \
> "$EVIDENCE_DIR/recent-deployments.json"
# Pull requests merged in audit period
gh pr list --repo YOUR_ORG/YOUR_REPO --state merged --limit 100 \
--json number,title,author,mergedBy,mergedAt,reviews \
> "$EVIDENCE_DIR/merged-prs.json"
echo "=== A1 - Availability Evidence ==="
# Backup status
aws rds describe-db-snapshots --db-instance-identifier prod-db \
--query 'DBSnapshots | sort_by(@, &SnapshotCreateTime) | [-5:]' \
> "$EVIDENCE_DIR/rds-backup-snapshots.json"
# S3 replication status
aws s3api get-bucket-replication --bucket prod-data-bucket \
> "$EVIDENCE_DIR/s3-replication-config.json"
echo "Evidence collected in $EVIDENCE_DIR"
tar -czf "$EVIDENCE_DIR.tar.gz" "$EVIDENCE_DIR"
echo "Archive: $EVIDENCE_DIR.tar.gz"
```
## Audit Preparation Timeline
```yaml
audit_prep_timeline:
12_months_before:
- Select auditor firm and sign engagement letter
- Perform gap assessment against TSC criteria
- Remediate identified control gaps
- Begin formal evidence collection cadence
6_months_before:
- Conduct internal readiness assessment
- Verify all controls are operating effectively
- Complete risk assessment and update risk register
- Ensure vendor assessments are current
- Test disaster recovery procedures
3_months_before:
- Run automated evidence collection and verify completeness
- Conduct access review and remediate findings
- Review and update all policies and procedures
- Perform vulnerability scan and penetration test
- Confirm all training records are current
1_month_before:
- Prepare evidence request list responses
- Organize evidence into auditor-friendly structure
- Brief key personnel on audit interviews
- Verify monitoring dashboards show healthy state
- Confirm incident response records are complete
during_audit:
- Designate audit liaison for request management
- Provide timely evidence and clarifications
- Track open auditor questions
- Escalate issues to control owners promptly
after_audit:
- Review draft report and provide management response
- Create remediation plan for any exceptions
- Communicate results to stakeholders
- Update controls and processes based on findings
- Begin next audit period evidence collection
```
## Continuous Compliance Monitoring
```yaml
# GitHub Actions workflow for continuous SOC 2 checks
name: SOC2 Compliance Checks
on:
schedule:
- cron: '0 6 * * 1' # Weekly on Monday
workflow_dispatch:
jobs:
access-review:
runs-on: ubuntu-latest
steps:
- name: Check MFA enforcement
run: |
USERS_WITHOUT_MFA=$(aws iam generate-credential-report && sleep 5 && \
aws iam get-credential-report --output text --query Content | \
base64 -d | awk -F, '$4=="true" && $8=="false" {print $1}')
if [ -n "$USERS_WITHOUT_MFA" ]; then
echo "::error::Users without MFA: $USERS_WITHOUT_MFA"
exit 1
fi
- name: Check for unused credentials
run: |
THRESHOLD=$(date -d '90 days ago' +%Y-%m-%dT%H:%M:%S)
aws iam get-credential-report --output text --query Content | \
base64 -d | awk -F, -v t="$THRESHOLD" '$5!="N/A" && $5<t {print $1" last used "$5}'
- name: Verify CloudTrail is logging
run: |
STATUS=$(aws cloudtrail get-trail-status --name org-audit-trail --query 'IsLogging' --output text)
[ "$STATUS" = "True" ] || (echo "::error::CloudTrail logging stopped" && exit 1)
- name: Check GuardDuty is enabled
run: |
DETECTOR=$(aws guardduty list-detectors --query 'DetectorIds[0]' --output text)
[ "$DETECTOR" != "None" ] || (echo "::error::GuardDuty not enabled" && exit 1)
```
## Best Practices
- Continuous compliance monitoring
- Annual risk assessments
- Regular control testing
- Documentation maintenance
- Start with a gap assessment to understand current control maturity before engaging an auditor
- Automate evidence collection to reduce the burden of auditor requests and ensure consistency
- Map each control to a specific tool, owner, and evidence artifact for traceability
- Implement continuous monitoring rather than point-in-time checks for Type II readiness
- Maintain a central evidence repository organized by control criteria
- Conduct quarterly internal reviews to catch control drift before the audit period
- Keep policies living documents with version history and annual review dates
- Train all employees on their role in maintaining SOC 2 controls
- Use the audit preparation timeline to avoid last-minute scrambling
- Treat each auditor exception as an improvement opportunity rather than a failure
compliance/governance/access-review/SKILL.md
+447 -44
View File
@@ -9,68 +9,471 @@ metadata:
# Access Review
Implement periodic access review processes.
Implement periodic access review processes for AWS IAM, GitHub, Okta, and other identity providers, including automated reporting, certification workflows, and unused permission detection.
## Review Process
## When to Use
- Conducting quarterly or annual access reviews for compliance (SOC 2, HIPAA, PCI DSS, ISO 27001)
- Identifying and removing stale accounts and unused credentials
- Certifying that current access levels match job responsibilities
- Detecting excessive privileges and dormant service accounts
- Generating evidence for auditor requests on access governance
## Access Review Process
```yaml
access_review_workflow:
1_extract:
- Pull access data from systems
- Generate access report
2_review:
- Manager certification
- Risk-based prioritization
- Decision documentation
3_action:
- Revoke unnecessary access
- Update exceptions
- Document decisions
4_report:
- Compliance metrics
- Remediation tracking
1_scope:
actions:
- Define systems in scope for the review cycle
- Identify review owners (managers, system owners)
- Set review timeline and deadlines
- Generate access inventory from all identity sources
frequency:
privileged_access: Quarterly
standard_access: Semi-annually
service_accounts: Quarterly
api_keys: Monthly
2_extract:
actions:
- Pull current access data from all systems
- Correlate identities across platforms (SSO mapping)
- Enrich with last login and activity data
- Flag accounts for review (inactive, over-privileged, orphaned)
3_review:
actions:
- Assign review items to appropriate managers
- Manager certifies each user's access (approve/revoke/modify)
- Risk-based prioritization (privileged users reviewed first)
- Escalate non-responses after deadline
decisions:
approve: "Access is appropriate for current role"
modify: "Access needs adjustment (reduce/change scope)"
revoke: "Access is no longer needed"
4_remediate:
actions:
- Revoke access flagged for removal
- Modify access as directed by reviewers
- Document exceptions with justification
- Confirm changes with system owners
sla:
revocations: "Complete within 5 business days of decision"
modifications: "Complete within 10 business days"
exceptions: "Approved by security team, documented, time-limited"
5_report:
actions:
- Generate completion metrics (% reviewed, % on time)
- Document all decisions and actions taken
- Archive evidence for compliance audits
- Identify process improvements for next cycle
```
## AWS IAM Review
## AWS IAM Access Review Scripts
```bash
#!/usr/bin/env bash
# aws-iam-review.sh - Comprehensive IAM access review report
OUTPUT_DIR="./access-review/$(date +%Y-%m)"
mkdir -p "$OUTPUT_DIR"
echo "=== AWS IAM Access Review ==="
# Generate credential report
aws iam generate-credential-report
aws iam get-credential-report --output text --query Content | base64 -d
aws iam generate-credential-report > /dev/null
sleep 10
aws iam get-credential-report --output text --query Content | \
base64 -d > "$OUTPUT_DIR/credential-report.csv"
# Find inactive users
aws iam list-users | jq -r '.Users[] | select(.PasswordLastUsed < "2024-01-01") | .UserName'
echo "--- Users Without MFA ---"
aws iam get-credential-report --output text --query Content | base64 -d | \
awk -F, 'NR>1 && $4=="true" && $8=="false" {print $1}' | \
tee "$OUTPUT_DIR/users-without-mfa.txt"
# List unused access keys
aws iam get-access-key-last-used --access-key-id AKIAXXXXXXXX
echo "--- Inactive Users (90+ days) ---"
THRESHOLD=$(date -d '90 days ago' +%Y-%m-%dT%H:%M:%S 2>/dev/null || date -v-90d +%Y-%m-%dT%H:%M:%S)
aws iam get-credential-report --output text --query Content | base64 -d | \
awk -F, -v t="$THRESHOLD" 'NR>1 && $5!="N/A" && $5!="no_information" && $5<t {
print $1","$5
}' | tee "$OUTPUT_DIR/inactive-users.csv"
echo "--- Stale Access Keys (90+ days unused) ---"
for user in $(aws iam list-users --query 'Users[*].UserName' --output text); do
for key_id in $(aws iam list-access-keys --user-name "$user" \
--query 'AccessKeyMetadata[?Status==`Active`].AccessKeyId' --output text); do
last_used=$(aws iam get-access-key-last-used --access-key-id "$key_id" \
--query 'AccessKeyLastUsed.LastUsedDate' --output text)
if [ "$last_used" = "None" ] || [ "$last_used" \< "$THRESHOLD" ]; then
echo "$user,$key_id,$last_used"
fi
done
done | tee "$OUTPUT_DIR/stale-access-keys.csv"
echo "--- Users With Admin Policies ---"
for user in $(aws iam list-users --query 'Users[*].UserName' --output text); do
policies=$(aws iam list-attached-user-policies --user-name "$user" \
--query 'AttachedPolicies[*].PolicyName' --output text)
if echo "$policies" | grep -qi "admin\|fullaccess"; then
groups=$(aws iam list-groups-for-user --user-name "$user" \
--query 'Groups[*].GroupName' --output text)
echo "$user|policies:$policies|groups:$groups"
fi
done | tee "$OUTPUT_DIR/admin-users.txt"
echo "--- IAM Roles With Cross-Account Trust ---"
for role in $(aws iam list-roles --query 'Roles[*].RoleName' --output text); do
trust=$(aws iam get-role --role-name "$role" \
--query 'Role.AssumeRolePolicyDocument' --output json 2>/dev/null)
if echo "$trust" | grep -q '"AWS"' && echo "$trust" | grep -qv "$(aws sts get-caller-identity --query Account --output text)"; then
echo "$role: $trust" | jq -c '.Statement[].Principal'
fi
done | tee "$OUTPUT_DIR/cross-account-roles.txt"
echo "--- Service Accounts (Programmatic Only) ---"
aws iam get-credential-report --output text --query Content | base64 -d | \
awk -F, 'NR>1 && $4=="false" && $9!="N/A" {print $1","$11","$16}' | \
tee "$OUTPUT_DIR/service-accounts.csv"
echo "Report generated in $OUTPUT_DIR"
```
## Automation
## GitHub Access Review
```bash
#!/usr/bin/env bash
# github-access-review.sh - GitHub organization access audit
ORG="your-org"
OUTPUT_DIR="./access-review/github/$(date +%Y-%m)"
mkdir -p "$OUTPUT_DIR"
echo "=== GitHub Organization Access Review ==="
echo "--- Organization Members ---"
gh api orgs/$ORG/members --paginate \
--jq '.[] | [.login, .site_admin] | @csv' \
> "$OUTPUT_DIR/org-members.csv"
echo "--- Organization Owners ---"
gh api "orgs/$ORG/members?role=admin" --paginate \
--jq '.[] | .login' \
> "$OUTPUT_DIR/org-owners.txt"
echo "--- Outside Collaborators ---"
gh api orgs/$ORG/outside_collaborators --paginate \
--jq '.[] | .login' \
> "$OUTPUT_DIR/outside-collaborators.txt"
echo "--- Repository Access Per Repo ---"
for repo in $(gh repo list $ORG --json name -q '.[].name' --limit 500); do
echo "Repo: $repo"
gh api "repos/$ORG/$repo/collaborators" --paginate \
--jq '.[] | [.login, .role_name] | @csv' \
> "$OUTPUT_DIR/repo-$repo-access.csv" 2>/dev/null
done
echo "--- Team Memberships ---"
for team in $(gh api orgs/$ORG/teams --paginate --jq '.[].slug'); do
echo "Team: $team"
gh api "orgs/$ORG/teams/$team/members" --paginate \
--jq '.[] | .login' \
> "$OUTPUT_DIR/team-$team-members.txt"
done
echo "--- Pending Invitations ---"
gh api orgs/$ORG/invitations --paginate \
--jq '.[] | [.login, .email, .role, .created_at] | @csv' \
> "$OUTPUT_DIR/pending-invitations.csv"
echo "--- Deploy Keys ---"
for repo in $(gh repo list $ORG --json name -q '.[].name' --limit 500); do
keys=$(gh api "repos/$ORG/$repo/keys" --jq '.[].title' 2>/dev/null)
if [ -n "$keys" ]; then
echo "$repo: $keys"
fi
done > "$OUTPUT_DIR/deploy-keys.txt"
echo "--- Branch Protection Rules ---"
for repo in $(gh repo list $ORG --json name -q '.[].name' --limit 500); do
protection=$(gh api "repos/$ORG/$repo/branches/main/protection" 2>/dev/null)
if [ $? -eq 0 ]; then
echo "$repo: protected"
echo "$protection" | jq '{required_reviews: .required_pull_request_reviews.required_approving_review_count, dismiss_stale: .required_pull_request_reviews.dismiss_stale_reviews}' \
> "$OUTPUT_DIR/branch-protection-$repo.json"
else
echo "$repo: NOT protected" >> "$OUTPUT_DIR/unprotected-repos.txt"
fi
done
echo "Report generated in $OUTPUT_DIR"
```
## Okta Access Review
```bash
#!/usr/bin/env bash
# okta-access-review.sh - Okta user and application access audit
# Requires OKTA_DOMAIN and OKTA_API_TOKEN environment variables
OUTPUT_DIR="./access-review/okta/$(date +%Y-%m)"
mkdir -p "$OUTPUT_DIR"
BASE_URL="https://${OKTA_DOMAIN}/api/v1"
echo "=== Okta Access Review ==="
echo "--- Active Users ---"
curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/users?filter=status+eq+%22ACTIVE%22&limit=200" | \
jq -r '.[] | [.profile.email, .profile.firstName, .profile.lastName, .lastLogin, .created] | @csv' \
> "$OUTPUT_DIR/active-users.csv"
echo "--- Suspended/Deprovisioned Users ---"
for status in SUSPENDED DEPROVISIONED; do
curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/users?filter=status+eq+%22$status%22&limit=200" | \
jq -r '.[] | [.profile.email, .status, .statusChanged] | @csv'
done > "$OUTPUT_DIR/inactive-users.csv"
echo "--- Users Without MFA Enrolled ---"
curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/users?limit=200" | \
jq -r '.[] | .id' | while read -r uid; do
factors=$(curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/users/$uid/factors" | jq 'length')
if [ "$factors" -eq 0 ]; then
curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/users/$uid" | jq -r '.profile.email'
fi
done > "$OUTPUT_DIR/users-without-mfa.txt"
echo "--- Application Assignments ---"
curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/apps?limit=200" | \
jq -r '.[] | [.id, .label, .status] | @csv' | while IFS=, read -r app_id app_name status; do
echo "App: $app_name"
curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/apps/$app_id/users?limit=200" | \
jq -r '.[] | [.credentials.userName // .profile.email, .status] | @csv'
done > "$OUTPUT_DIR/app-assignments.csv"
echo "--- Admin Role Assignments ---"
curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/users?limit=200" | \
jq -r '.[] | .id' | while read -r uid; do
roles=$(curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/users/$uid/roles" | jq -r '.[].type' 2>/dev/null)
if [ -n "$roles" ]; then
email=$(curl -s -H "Authorization: SSWS $OKTA_API_TOKEN" \
"$BASE_URL/users/$uid" | jq -r '.profile.email')
echo "$email: $roles"
fi
done > "$OUTPUT_DIR/admin-roles.txt"
echo "Report generated in $OUTPUT_DIR"
```
## Unused Permission Detection
```python
def generate_access_report():
users = get_all_users()
report = []
"""
Detect unused IAM permissions using CloudTrail and IAM Access Analyzer.
Generates recommendations for right-sizing access.
"""
import boto3
import json
import time
from datetime import datetime, timedelta, timezone
def analyze_iam_usage(days_lookback=90):
"""Analyze IAM user and role activity against granted permissions."""
iam = boto3.client("iam")
report = {
"generated_at": datetime.now(timezone.utc).isoformat(),
"lookback_days": days_lookback,
"findings": [],
}
users = iam.list_users()["Users"]
for user in users:
report.append({
'user': user.email,
'roles': user.roles,
'last_login': user.last_login,
'manager': user.manager,
'review_status': 'pending'
})
username = user["UserName"]
# Get service last accessed data
job_id = iam.generate_service_last_accessed_details(
Arn=user["Arn"]
)["JobId"]
while True:
result = iam.get_service_last_accessed_details(JobId=job_id)
if result["JobStatus"] == "COMPLETED":
break
time.sleep(2)
threshold = datetime.now(timezone.utc) - timedelta(days=days_lookback)
unused_services = []
for service in result["ServicesLastAccessed"]:
last_accessed = service.get("LastAuthenticated")
if last_accessed is None or last_accessed < threshold:
unused_services.append({
"service": service["ServiceNamespace"],
"last_accessed": str(last_accessed) if last_accessed else "Never",
})
if unused_services:
report["findings"].append({
"type": "unused_permissions",
"user": username,
"arn": user["Arn"],
"unused_service_count": len(unused_services),
"unused_services": unused_services[:10],
"recommendation": "Review and remove unused service permissions",
})
return report
def detect_overprivileged_roles():
"""Use IAM Access Analyzer to find overprivileged roles."""
analyzer = boto3.client("accessanalyzer")
findings = analyzer.list_findings(
analyzerArn="arn:aws:access-analyzer:us-east-1:123456789012:analyzer/org-analyzer",
filter={
"status": {"eq": ["ACTIVE"]},
"resourceType": {"eq": ["AWS::IAM::Role"]},
},
)
return [
{
"resource": f["resource"],
"resource_type": f["resourceType"],
"condition": f.get("condition", {}),
"principal": f.get("principal", {}),
"action": f.get("action", []),
"created_at": str(f["createdAt"]),
}
for f in findings.get("findings", [])
]
```
## Certification Workflow Automation
```yaml
# GitHub Actions - Automated access review reminder and tracking
name: Quarterly Access Review
on:
schedule:
- cron: '0 9 1 1,4,7,10 *' # First day of each quarter
workflow_dispatch:
jobs:
generate-review:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Generate access reports
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AUDIT_AWS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AUDIT_AWS_SECRET }}
OKTA_DOMAIN: ${{ secrets.OKTA_DOMAIN }}
OKTA_API_TOKEN: ${{ secrets.OKTA_API_TOKEN }}
run: |
bash scripts/aws-iam-review.sh
bash scripts/github-access-review.sh
bash scripts/okta-access-review.sh
- name: Create review issue
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
QUARTER="Q$(( ($(date +%-m) - 1) / 3 + 1 )) $(date +%Y)"
MFA_MISSING=$(wc -l < access-review/$(date +%Y-%m)/users-without-mfa.txt)
INACTIVE=$(wc -l < access-review/$(date +%Y-%m)/inactive-users.csv)
STALE_KEYS=$(wc -l < access-review/$(date +%Y-%m)/stale-access-keys.csv)
gh issue create \
--title "Access Review - $QUARTER" \
--label "compliance,access-review" \
--body "## Quarterly Access Review - $QUARTER
### Summary
- Users without MFA: **$MFA_MISSING**
- Inactive users (90+ days): **$INACTIVE**
- Stale access keys: **$STALE_KEYS**
### Required Actions
- [ ] Review and disable inactive users
- [ ] Enforce MFA for non-compliant users
- [ ] Rotate or deactivate stale access keys
- [ ] Review admin/privileged access assignments
- [ ] Review outside collaborators on GitHub
- [ ] Certify remaining access is appropriate
- [ ] Document exceptions with justification
### Deadline
Complete within 30 days."
- name: Upload reports as artifact
uses: actions/upload-artifact@v4
with:
name: access-review-reports
path: access-review/
retention-days: 365
```
## Access Review Checklist
```yaml
access_review_checklist:
preparation:
- [ ] Define scope (systems, user populations, review period)
- [ ] Assign review owners for each system
- [ ] Extract current access data from all identity sources
- [ ] Correlate identities across platforms via SSO mapping
- [ ] Generate review packages for each manager
execution:
- [ ] Managers notified with review assignments and deadline
- [ ] Privileged access reviewed first (admin, root, service accounts)
- [ ] Each user's access certified (approve, modify, or revoke)
- [ ] Inactive accounts flagged for disable/removal
- [ ] Stale credentials (keys, tokens) flagged for rotation
- [ ] Outside collaborators and contractors verified
- [ ] Service account ownership confirmed
remediation:
- [ ] Revocations executed within SLA (5 business days)
- [ ] Access modifications completed within SLA (10 business days)
- [ ] Exceptions documented with business justification
- [ ] Exception approvals recorded from security team
- [ ] Changes verified in target systems
reporting:
- [ ] Review completion rate documented (target: 100%)
- [ ] Non-response escalations documented
- [ ] Remediation actions summarized
- [ ] Exception register updated
- [ ] Evidence archived for audit (retained 3+ years)
- [ ] Metrics compared to prior review cycle
```
## Best Practices
- Quarterly reviews minimum
- Risk-based frequency
- Manager attestation
- Automated revocation
- Audit trail maintenance
- Automate access data extraction to eliminate manual data gathering and reduce errors
- Integrate access review with HR systems to automatically flag accounts for departed employees
- Use risk-based review frequency: privileged access quarterly, standard access semi-annually
- Provide managers with clear context: show last login date, permissions, and role to inform decisions
- Set firm deadlines with escalation for non-response (no certification = automatic revocation)
- Detect and eliminate orphaned accounts from contractors, former employees, and decommissioned services
- Review service accounts and API keys alongside human accounts to prevent credential sprawl
- Document all exceptions with business justification, approver, and expiration date
- Track review metrics over time: completion rates, revocation rates, time to remediate
- Archive all access review evidence for a minimum of 3 years for audit purposes
compliance/governance/asset-inventory/SKILL.md
+475 -46
View File
@@ -9,66 +9,495 @@ metadata:
# Asset Inventory
Maintain comprehensive IT asset tracking.
Maintain comprehensive IT asset inventory using automated discovery, AWS Config rules, cloud asset discovery scripts, CMDB integration, and tagging enforcement for compliance and operational visibility.
## Asset Categories
## When to Use
- Building or maintaining an IT asset inventory for compliance frameworks (ISO 27001, SOC 2, FedRAMP)
- Implementing automated cloud resource discovery across accounts and regions
- Enforcing tagging standards for cost allocation, ownership, and data classification
- Integrating asset data with a CMDB for operational workflows
- Preparing for audits that require a complete system component inventory
## Asset Categories and Schema
```yaml
asset_types:
hardware:
- Servers
- Network devices
- Endpoints
software:
- Applications
- Operating systems
- Licenses
cloud:
- Compute instances
- Storage
- Databases
data:
- Databases
- File shares
- Backups
asset_categories:
compute:
cloud:
- EC2 instances / Azure VMs / GCE instances
- Lambda functions / Azure Functions / Cloud Functions
- ECS/EKS clusters and tasks
- Container images in registries
on_premise:
- Physical servers
- Virtual machines (VMware, Hyper-V)
storage:
- S3 buckets / Azure Storage / GCS buckets
- EBS volumes / Managed Disks / Persistent Disks
- RDS instances / Azure SQL / Cloud SQL
- DynamoDB tables / Cosmos DB / Firestore
- EFS / Azure Files / Filestore
network:
- VPCs / VNets / VPC Networks
- Load balancers (ALB, NLB, Azure LB, GCP LB)
- DNS zones and records
- VPN gateways and connections
- CDN distributions
security:
- IAM users, roles, and policies
- KMS keys / Key Vault keys
- Certificates (ACM, Key Vault, Certificate Manager)
- Security groups / NSGs / Firewall rules
- WAF configurations
applications:
- SaaS subscriptions
- Licensed software
- Custom applications
- APIs and integrations
endpoints:
- Laptops and desktops
- Mobile devices
- Printers and peripherals
asset_record_schema:
required_fields:
asset_id: "Unique identifier (auto-generated)"
name: "Human-readable name"
type: "Category from above taxonomy"
provider: "AWS / Azure / GCP / On-Premise / SaaS"
account_or_subscription: "Cloud account ID"
region: "Deployment region/location"
owner: "Team or individual responsible"
data_classification: "Public / Internal / Confidential / Restricted"
environment: "Production / Staging / Development / Sandbox"
status: "Active / Decommissioning / Retired"
created_date: "When the asset was provisioned"
last_seen: "Last automated discovery timestamp"
optional_fields:
cost_center: "For cost allocation"
compliance_scope: "SOC2 / HIPAA / PCI / None"
backup_policy: "Backup schedule reference"
dr_tier: "Critical / Essential / Standard / Non-essential"
expiration_date: "For time-limited resources"
tags: "Key-value pairs from cloud provider"
dependencies: "Upstream and downstream services"
```
## AWS Inventory
## AWS Resource Discovery Script
```bash
# List all resources
aws resourcegroupstaggingapi get-resources
#!/usr/bin/env bash
# aws-asset-discovery.sh - Discover and inventory all AWS resources
# EC2 instances
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId,InstanceType,State.Name]'
OUTPUT_DIR="./asset-inventory/aws/$(date +%Y-%m-%d)"
mkdir -p "$OUTPUT_DIR"
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
# AWS Config
aws configservice describe-configuration-recorders
echo "=== AWS Asset Discovery for Account $ACCOUNT_ID ==="
# EC2 Instances
echo "--- EC2 Instances ---"
aws ec2 describe-instances \
--query 'Reservations[*].Instances[*].{
InstanceId:InstanceId,
Type:InstanceType,
State:State.Name,
AZ:Placement.AvailabilityZone,
VpcId:VpcId,
PrivateIP:PrivateIpAddress,
PublicIP:PublicIpAddress,
LaunchTime:LaunchTime,
Name:Tags[?Key==`Name`].Value|[0],
Owner:Tags[?Key==`Owner`].Value|[0],
Environment:Tags[?Key==`Environment`].Value|[0]
}' --output json | jq 'flatten' > "$OUTPUT_DIR/ec2-instances.json"
# RDS Databases
echo "--- RDS Instances ---"
aws rds describe-db-instances \
--query 'DBInstances[*].{
DBInstanceId:DBInstanceIdentifier,
Engine:Engine,
EngineVersion:EngineVersion,
Class:DBInstanceClass,
Status:DBInstanceStatus,
MultiAZ:MultiAZ,
Encrypted:StorageEncrypted,
Endpoint:Endpoint.Address,
BackupRetention:BackupRetentionPeriod
}' --output json > "$OUTPUT_DIR/rds-instances.json"
# S3 Buckets
echo "--- S3 Buckets ---"
aws s3api list-buckets --query 'Buckets[*].{Name:Name,Created:CreationDate}' --output json | \
jq -c '.[]' | while read -r bucket; do
name=$(echo "$bucket" | jq -r '.Name')
region=$(aws s3api get-bucket-location --bucket "$name" --query 'LocationConstraint' --output text 2>/dev/null)
encryption=$(aws s3api get-bucket-encryption --bucket "$name" 2>/dev/null | jq -r '.ServerSideEncryptionConfiguration.Rules[0].ApplyServerSideEncryptionByDefault.SSEAlgorithm' 2>/dev/null)
versioning=$(aws s3api get-bucket-versioning --bucket "$name" --query 'Status' --output text 2>/dev/null)
echo "{\"Name\":\"$name\",\"Region\":\"${region:-us-east-1}\",\"Encryption\":\"${encryption:-none}\",\"Versioning\":\"${versioning:-Disabled}\"}"
done | jq -s '.' > "$OUTPUT_DIR/s3-buckets.json"
# Lambda Functions
echo "--- Lambda Functions ---"
aws lambda list-functions \
--query 'Functions[*].{
Name:FunctionName,
Runtime:Runtime,
MemorySize:MemorySize,
Timeout:Timeout,
LastModified:LastModified,
CodeSize:CodeSize
}' --output json > "$OUTPUT_DIR/lambda-functions.json"
# VPCs and Security Groups
echo "--- VPCs ---"
aws ec2 describe-vpcs \
--query 'Vpcs[*].{
VpcId:VpcId,
CidrBlock:CidrBlock,
State:State,
IsDefault:IsDefault,
Name:Tags[?Key==`Name`].Value|[0]
}' --output json > "$OUTPUT_DIR/vpcs.json"
echo "--- Security Groups ---"
aws ec2 describe-security-groups \
--query 'SecurityGroups[*].{
GroupId:GroupId,
GroupName:GroupName,
VpcId:VpcId,
Description:Description,
IngressRuleCount:length(IpPermissions),
EgressRuleCount:length(IpPermissionsEgress)
}' --output json > "$OUTPUT_DIR/security-groups.json"
# IAM Users and Roles
echo "--- IAM Users ---"
aws iam list-users \
--query 'Users[*].{UserName:UserName,Created:CreateDate,PasswordLastUsed:PasswordLastUsed}' \
--output json > "$OUTPUT_DIR/iam-users.json"
echo "--- IAM Roles ---"
aws iam list-roles \
--query 'Roles[*].{RoleName:RoleName,Created:CreateDate,LastUsed:RoleLastUsed.LastUsedDate}' \
--output json > "$OUTPUT_DIR/iam-roles.json"
# EKS Clusters
echo "--- EKS Clusters ---"
aws eks list-clusters --query 'clusters' --output json | jq -r '.[]' | while read -r cluster; do
aws eks describe-cluster --name "$cluster" \
--query 'cluster.{Name:name,Version:version,Status:status,Endpoint:endpoint,Created:createdAt}'
done | jq -s '.' > "$OUTPUT_DIR/eks-clusters.json" 2>/dev/null
# KMS Keys
echo "--- KMS Keys ---"
aws kms list-keys --query 'Keys[*].KeyId' --output text | tr '\t' '\n' | while read -r key_id; do
aws kms describe-key --key-id "$key_id" \
--query 'KeyMetadata.{KeyId:KeyId,Description:Description,State:KeyState,Created:CreationDate,Manager:KeyManager}' 2>/dev/null
done | jq -s '.' > "$OUTPUT_DIR/kms-keys.json"
# Generate summary
echo "=== Inventory Summary ==="
echo "EC2 Instances: $(jq 'length' "$OUTPUT_DIR/ec2-instances.json")"
echo "RDS Instances: $(jq 'length' "$OUTPUT_DIR/rds-instances.json")"
echo "S3 Buckets: $(jq 'length' "$OUTPUT_DIR/s3-buckets.json")"
echo "Lambda Functions: $(jq 'length' "$OUTPUT_DIR/lambda-functions.json")"
echo "VPCs: $(jq 'length' "$OUTPUT_DIR/vpcs.json")"
echo "Security Groups: $(jq 'length' "$OUTPUT_DIR/security-groups.json")"
echo "IAM Users: $(jq 'length' "$OUTPUT_DIR/iam-users.json")"
echo "IAM Roles: $(jq 'length' "$OUTPUT_DIR/iam-roles.json")"
echo "Inventory saved to $OUTPUT_DIR"
```
## Asset Database Schema
## AWS Config Rules for Inventory Compliance
```bash
# Enable AWS Config recorder
aws configservice put-configuration-recorder \
--configuration-recorder name=default,roleARN=arn:aws:iam::123456789012:role/aws-config-role \
--recording-group allSupported=true,includeGlobalResourceTypes=true
# Start recording
aws configservice start-configuration-recorder --configuration-recorder-name default
# Enable required-tags Config rule
aws configservice put-config-rule --config-rule '{
"ConfigRuleName": "required-tags",
"Source": {
"Owner": "AWS",
"SourceIdentifier": "REQUIRED_TAGS"
},
"InputParameters": "{\"tag1Key\":\"Owner\",\"tag2Key\":\"Environment\",\"tag3Key\":\"CostCenter\",\"tag4Key\":\"DataClassification\"}",
"Scope": {
"ComplianceResourceTypes": [
"AWS::EC2::Instance",
"AWS::RDS::DBInstance",
"AWS::S3::Bucket",
"AWS::Lambda::Function"
]
}
}'
# Config rule for encryption compliance
aws configservice put-config-rule --config-rule '{
"ConfigRuleName": "encrypted-volumes",
"Source": {
"Owner": "AWS",
"SourceIdentifier": "ENCRYPTED_VOLUMES"
}
}'
aws configservice put-config-rule --config-rule '{
"ConfigRuleName": "rds-storage-encrypted",
"Source": {
"Owner": "AWS",
"SourceIdentifier": "RDS_STORAGE_ENCRYPTED"
}
}'
aws configservice put-config-rule --config-rule '{
"ConfigRuleName": "s3-bucket-server-side-encryption-enabled",
"Source": {
"Owner": "AWS",
"SourceIdentifier": "S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED"
}
}'
# Query AWS Config for all resources of a type
aws configservice list-discovered-resources --resource-type AWS::EC2::Instance
aws configservice list-discovered-resources --resource-type AWS::RDS::DBInstance
# Advanced query with AWS Config SQL
aws configservice select-resource-config \
--expression "SELECT resourceId, resourceType, tags, configuration.instanceType
WHERE resourceType = 'AWS::EC2::Instance'
AND tags.tag('Environment') = 'production'"
# Get compliance summary
aws configservice get-compliance-summary-by-config-rule
aws configservice get-compliance-summary-by-resource-type
```
## Tagging Enforcement
```yaml
asset:
id: unique identifier
name: display name
type: hardware/software/cloud
owner: responsible team
classification: public/internal/confidential
location: physical/cloud location
status: active/retired/decommissioned
created: timestamp
updated: timestamp
tags: []
# AWS Tag Policy (applied via AWS Organizations)
tag_policy:
tags:
Owner:
tag_key:
"@@assign": "Owner"
enforced_for:
"@@assign":
- "ec2:instance"
- "rds:db"
- "s3:bucket"
- "lambda:function"
Environment:
tag_key:
"@@assign": "Environment"
tag_value:
"@@assign":
- "production"
- "staging"
- "development"
- "sandbox"
DataClassification:
tag_key:
"@@assign": "DataClassification"
tag_value:
"@@assign":
- "public"
- "internal"
- "confidential"
- "restricted"
CostCenter:
tag_key:
"@@assign": "CostCenter"
```
```hcl
# Terraform - Enforce tags on all resources with default_tags
provider "aws" {
region = "us-east-1"
default_tags {
tags = {
ManagedBy = "terraform"
Environment = var.environment
Owner = var.team_name
CostCenter = var.cost_center
DataClassification = var.data_classification
}
}
}
```
## Multi-Cloud Discovery
```bash
#!/usr/bin/env bash
# multi-cloud-discovery.sh - Discover assets across AWS, Azure, and GCP
OUTPUT_DIR="./asset-inventory/multi-cloud/$(date +%Y-%m-%d)"
mkdir -p "$OUTPUT_DIR"
echo "=== Multi-Cloud Asset Discovery ==="
# AWS - using Resource Groups Tagging API
echo "--- AWS Resources ---"
aws resourcegroupstaggingapi get-resources \
--query 'ResourceTagMappingList[*].{ARN:ResourceARN,Tags:Tags}' \
--output json > "$OUTPUT_DIR/aws-all-resources.json"
echo "AWS resources: $(jq 'length' "$OUTPUT_DIR/aws-all-resources.json")"
# Azure - using Resource Graph
echo "--- Azure Resources ---"
az graph query -q "Resources | project name, type, location, resourceGroup, subscriptionId, tags" \
--output json > "$OUTPUT_DIR/azure-all-resources.json" 2>/dev/null
# GCP - using Cloud Asset Inventory
echo "--- GCP Resources ---"
gcloud asset search-all-resources \
--scope="organizations/ORG_ID" \
--format=json > "$OUTPUT_DIR/gcp-all-resources.json" 2>/dev/null
# Find untagged resources
echo "=== Untagged Resources ==="
jq '[.[] | select(.Tags == null or .Tags == [])] | length' "$OUTPUT_DIR/aws-all-resources.json"
echo "Discovery complete. Results in $OUTPUT_DIR"
```
## CMDB Integration
```python
"""
CMDB sync script - Normalize cloud assets and push to CMDB API.
"""
import json
import requests
from datetime import datetime, timezone
class CMDBSync:
def __init__(self, cmdb_url, api_token):
self.cmdb_url = cmdb_url
self.headers = {
"Authorization": f"Bearer {api_token}",
"Content-Type": "application/json",
}
def normalize_aws_instance(self, instance):
"""Convert AWS EC2 instance to common asset schema."""
tags = {t["Key"]: t["Value"] for t in (instance.get("Tags") or [])}
return {
"asset_id": f"aws:{instance['InstanceId']}",
"name": tags.get("Name", instance["InstanceId"]),
"type": "compute",
"provider": "aws",
"region": instance.get("AZ", "unknown")[:-1],
"configuration": {
"instance_type": instance.get("Type"),
"state": instance.get("State"),
"vpc_id": instance.get("VpcId"),
},
"owner": tags.get("Owner", "unassigned"),
"environment": tags.get("Environment", "unknown"),
"data_classification": tags.get("DataClassification", "unknown"),
"status": "active" if instance.get("State") == "running" else "stopped",
"last_seen": datetime.now(timezone.utc).isoformat(),
}
def sync_assets(self, assets):
"""Push normalized assets to CMDB."""
results = {"created": 0, "updated": 0, "errors": 0}
for asset in assets:
try:
resp = requests.get(
f"{self.cmdb_url}/assets/{asset['asset_id']}",
headers=self.headers,
)
if resp.status_code == 200:
requests.put(
f"{self.cmdb_url}/assets/{asset['asset_id']}",
headers=self.headers,
json=asset,
)
results["updated"] += 1
else:
requests.post(
f"{self.cmdb_url}/assets",
headers=self.headers,
json=asset,
)
results["created"] += 1
except Exception:
results["errors"] += 1
return results
```
## Asset Inventory Checklist
```yaml
asset_inventory_checklist:
discovery:
- [ ] Automated discovery scripts running for all cloud accounts
- [ ] Discovery covers all resource types (compute, storage, network, IAM)
- [ ] Multi-region discovery enabled
- [ ] On-premise assets cataloged
- [ ] SaaS subscriptions inventoried
- [ ] Discovery runs daily (minimum weekly)
classification:
- [ ] Required tags defined (Owner, Environment, DataClassification, CostCenter)
- [ ] Tag enforcement via AWS Organizations tag policies
- [ ] Tag enforcement via Terraform default_tags
- [ ] Tag enforcement via CI/CD policy checks (Checkov, OPA)
- [ ] Untagged resource reports generated and tracked
configuration_management:
- [ ] AWS Config enabled in all regions
- [ ] Config rules enforce encryption, tagging, and security baselines
- [ ] Configuration compliance summary reviewed weekly
- [ ] Drift detection enabled for IaC-managed resources
cmdb:
- [ ] CMDB sync automated from cloud discovery
- [ ] Common schema defined across all providers
- [ ] Reconciliation process identifies orphaned records
- [ ] New resources auto-assigned default owner
- [ ] Asset lifecycle tracked (created, active, decommissioning, retired)
governance:
- [ ] Asset owners assigned and current
- [ ] Quarterly inventory reconciliation conducted
- [ ] Compliance scope tagging accurate (SOC2, HIPAA, PCI)
- [ ] Asset inventory available for auditor review
- [ ] Decommissioned assets tracked for data retention compliance
```
## Best Practices
- Automated discovery
- Regular reconciliation
- Owner assignment
- Classification tagging
- Lifecycle tracking
- Automate discovery rather than relying on manual inventory: cloud environments change too fast for spreadsheets
- Use AWS Config, Azure Resource Graph, and GCP Cloud Asset Inventory as authoritative data sources
- Enforce tagging at provisioning time through IaC defaults and policy-as-code guardrails
- Assign every asset an owner: unowned resources become security and cost liabilities
- Reconcile inventory regularly and investigate orphaned assets (CMDB record with no real resource and vice versa)
- Track data classification as a mandatory tag to support compliance scoping decisions
- Maintain asset lifecycle states to distinguish active resources from those being decommissioned
- Integrate asset inventory with incident response to quickly identify affected systems during investigations
- Export inventory data for compliance audits in accessible formats (CSV, JSON)
- Review untagged and unclassified resource reports weekly to maintain inventory quality
compliance/governance/change-management/SKILL.md
+475 -51
View File
@@ -9,73 +9,497 @@ metadata:
# Change Management
Implement structured change management processes.
Implement structured change management processes covering change classification, CAB workflows, emergency change procedures, and automation for compliance with SOC 2, ITIL, and regulatory frameworks.
## Change Process
## When to Use
```yaml
change_workflow:
1_request:
- Change description
- Risk assessment
- Rollback plan
- Testing evidence
2_review:
- Technical review
- Security review
- CAB approval (if high risk)
3_schedule:
- Change window
- Communication
- Resource allocation
4_implement:
- Execute change
- Verify success
- Update documentation
5_review:
- Post-implementation review
- Lessons learned
```
- Establishing change management processes for production environments
- Implementing change advisory board (CAB) workflows
- Defining change classification and approval requirements
- Configuring automated change tracking in CI/CD pipelines
- Handling emergency changes with proper controls and documentation
## Change Classification
| Type | Risk | Approval | Example |
|------|------|----------|---------|
| Standard | Low | Pre-approved | Patching |
| Normal | Medium | Manager | Config change |
| Emergency | Variable | Expedited | Security fix |
```yaml
change_types:
standard:
risk: Low
approval: Pre-approved (no per-change approval needed)
lead_time: None (within maintenance window)
examples:
- Routine patching within tested patch sets
- Certificate rotation with established procedure
- Scaling operations (adding/removing instances within limits)
- Pre-approved configuration changes
- Log rotation and archival
requirements:
- Change must match an approved Standard Change template
- Automated testing must pass
- Documented rollback procedure exists
- Within defined maintenance window
## Pull Request Template
normal_low:
risk: Low
approval: Peer review (1 approver)
lead_time: 2 business days
examples:
- Non-critical configuration changes
- Feature flag toggles
- Documentation updates to production systems
- Adding monitoring dashboards or alerts
normal_medium:
risk: Medium
approval: Team lead + peer review (2 approvers)
lead_time: 5 business days
examples:
- Application deployments with new features
- Database schema changes (non-breaking)
- Network rule modifications
- Integration endpoint changes
- Dependency version upgrades
normal_high:
risk: High
approval: CAB review required
lead_time: 10 business days
examples:
- Infrastructure migrations
- Breaking database schema changes
- Major version upgrades (OS, runtime, database engine)
- Changes to authentication or authorization systems
- Multi-service coordinated deployments
- Changes affecting data processing or compliance controls
emergency:
risk: Variable
approval: Emergency CAB (minimum 2 approvers from on-call)
lead_time: None (immediate implementation)
examples:
- Security vulnerability remediation (active exploitation)
- Production outage resolution
- Data integrity emergency fixes
- Regulatory compliance deadline fixes
requirements:
- Retroactive full documentation within 48 hours
- Post-implementation review required
- CAB retroactive review at next meeting
```
## Change Request Template
```yaml
change_request:
metadata:
id: "CR-YYYY-NNNN"
title: ""
requestor: ""
date_submitted: ""
target_date: ""
change_type: "" # standard | normal_low | normal_medium | normal_high | emergency
description:
summary: "Brief description of the change"
detailed_description: "Full technical details of what will change"
business_justification: "Why this change is needed"
affected_systems: []
affected_services: []
affected_users: "Description of user impact"
risk_assessment:
risk_level: "" # low | medium | high
impact_if_failed: "What happens if the change fails"
likelihood_of_failure: "" # low | medium | high
risk_mitigation: "Steps to reduce risk"
dependencies: "Other systems or changes this depends on"
implementation:
change_window:
start: ""
end: ""
maintenance_window: true
implementation_steps:
- step: "Step 1 description"
responsible: "Person/team"
estimated_duration: "X minutes"
- step: "Step 2 description"
responsible: "Person/team"
estimated_duration: "X minutes"
testing:
pre_change_testing:
- "Unit tests pass"
- "Integration tests pass"
- "Staging deployment verified"
post_change_verification:
- "Health check endpoints responding"
- "Key transactions processing successfully"
- "No error rate increase in monitoring"
- "Performance metrics within baseline"
rollback:
rollback_plan: "Detailed steps to revert the change"
rollback_trigger: "Conditions that trigger rollback"
rollback_estimated_time: "X minutes"
rollback_steps:
- "Step 1: Revert deployment to previous version"
- "Step 2: Verify rollback successful"
- "Step 3: Notify stakeholders"
data_rollback: "Describe any data migration rollback needed"
communication:
stakeholders_notified: []
notification_sent_date: ""
status_page_update: true
customer_notification_required: false
approvals:
technical_reviewer: ""
technical_approval_date: ""
security_reviewer: ""
security_approval_date: ""
cab_approval_date: ""
cab_notes: ""
closure:
implementation_date: ""
implementation_result: "" # success | partial | failed | rolled_back
post_implementation_review: ""
lessons_learned: ""
follow_up_actions: []
```
## CAB Workflow
```yaml
cab_workflow:
meeting_schedule:
regular_cab: "Weekly, Thursday 2:00 PM"
emergency_cab: "On-demand, minimum 2 members required"
cab_members:
permanent:
- Engineering Manager (Chair)
- Security Team Representative
- Infrastructure/SRE Lead
- Release Manager
advisory:
- Business stakeholder (invited per change)
- Database administrator (for DB changes)
- Network engineer (for network changes)
agenda:
1: "Review emergency changes from prior week"
2: "Review high-risk change requests for upcoming window"
3: "Review failed changes and lessons learned"
4: "Discuss upcoming change freeze periods"
5: "Review change metrics and trends"
decision_criteria:
approve_when:
- Risk assessment is complete and accurate
- Testing evidence is provided
- Rollback plan is documented and feasible
- Change window is appropriate
- Required approvals obtained
- No conflicts with other scheduled changes
request_changes_when:
- Rollback plan is missing or incomplete
- Testing is insufficient for the risk level
- Impact assessment needs clarification
- Change conflicts with another scheduled change
deny_when:
- Risk is unacceptable without mitigation
- Change window conflicts with freeze period
- Dependencies are not resolved
- Compliance concerns are unaddressed
```
## Emergency Change Procedure
```yaml
emergency_change_process:
definition: "A change required to restore service or prevent imminent security compromise"
step_1_declare:
actions:
- On-call engineer identifies need for emergency change
- Incident commander approves emergency classification
- Minimum 2 approvers from emergency CAB roster contacted
- Document initial justification in incident channel
step_2_approve:
approval_method:
- Slack/Teams approval with screenshots preserved
- Verbal approval over bridge call (documented in notes)
- Emergency approvers can be any 2 of the following roles:
- Engineering Manager
- SRE/Infrastructure Lead
- Security Team Lead
- VP of Engineering
timeout: "If no response in 15 minutes, escalate to next tier"
step_3_implement:
actions:
- Implement the minimum change needed to resolve the issue
- Record all actions taken with timestamps
- Monitor for successful resolution
- Document any deviations from planned change
step_4_verify:
actions:
- Confirm service restoration
- Verify no unintended side effects
- Run post-change verification checks
- Update status page and stakeholders
step_5_document:
deadline: "Within 48 hours of implementation"
required_documentation:
- Complete change request form (retroactive)
- Timeline of events and actions
- Justification for emergency classification
- Approval records (messages, emails)
- Post-implementation verification results
- Root cause analysis (what made it an emergency)
- Preventive actions to avoid future emergency
step_6_review:
actions:
- CAB review at next regular meeting
- Assess if emergency classification was appropriate
- Identify process improvements
- Track emergency change trends
```
## Pull Request Template for Changes
```markdown
## Change Description
## Change Request
## Risk Level
- [ ] Low - Standard change
- [ ] Medium - Normal change
- [ ] High - CAB required
### Type
- [ ] Standard (pre-approved, low risk)
- [ ] Normal - Low Risk
- [ ] Normal - Medium Risk
- [ ] Normal - High Risk (CAB required)
- [ ] Emergency (retroactive documentation required)
## Testing
### Description
<!-- What is being changed and why? -->
### Risk Assessment
**Impact if failed:** <!-- What breaks? -->
**Likelihood of failure:** Low / Medium / High
**Affected services:** <!-- List services -->
**User impact:** <!-- Will users notice? -->
### Testing Evidence
- [ ] Unit tests pass
- [ ] Integration tests pass
- [ ] Staging deployment verified
- [ ] Performance test completed (if applicable)
- [ ] Security scan clean (if applicable)
## Rollback Plan
### Rollback Plan
<!-- How to revert if something goes wrong -->
**Estimated rollback time:** <!-- X minutes -->
**Data rollback needed:** Yes / No
## Stakeholders Notified
- [ ] Operations
- [ ] Security
- [ ] Business owners
### Deployment Plan
**Target window:** <!-- Date and time -->
**Estimated duration:** <!-- X minutes -->
### Post-Deployment Verification
- [ ] Health checks passing
- [ ] Error rates within baseline
- [ ] Key transactions working
- [ ] Monitoring dashboards reviewed
### Communication
- [ ] Team notified
- [ ] Stakeholders notified (if user-facing)
- [ ] Status page updated (if applicable)
### Approvals Required
- [ ] Peer review
- [ ] Team lead (medium+ risk)
- [ ] Security review (security-impacting changes)
- [ ] CAB approval (high risk)
```
## CI/CD Change Tracking Automation
```yaml
# GitHub Actions - Automated change tracking
name: Change Management
on:
pull_request:
types: [opened, synchronize, labeled]
push:
branches: [main]
jobs:
classify-change:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Classify change risk
id: classify
run: |
FILES_CHANGED=$(gh pr diff ${{ github.event.pull_request.number }} --name-only)
# High risk indicators
if echo "$FILES_CHANGED" | grep -qE 'terraform/|infrastructure/|migrations/|auth/|security/'; then
echo "risk=high" >> $GITHUB_OUTPUT
echo "::warning::High-risk change detected - CAB review may be required"
elif echo "$FILES_CHANGED" | grep -qE 'config/|database/|api/'; then
echo "risk=medium" >> $GITHUB_OUTPUT
else
echo "risk=low" >> $GITHUB_OUTPUT
fi
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Add risk label
run: |
gh pr edit ${{ github.event.pull_request.number }} \
--add-label "risk:${{ steps.classify.outputs.risk }}"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Enforce approvals by risk
if: steps.classify.outputs.risk == 'high'
run: |
APPROVALS=$(gh pr view ${{ github.event.pull_request.number }} \
--json reviews --jq '[.reviews[] | select(.state=="APPROVED")] | length')
if [ "$APPROVALS" -lt 2 ]; then
echo "::error::High-risk changes require at least 2 approvals"
exit 1
fi
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
record-deployment:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- name: Record deployment
run: |
CHANGE_ID="CR-$(date +%Y)-$(printf '%04d' ${{ github.run_number }})"
echo "Change ID: $CHANGE_ID"
echo "Deployed at: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo "Commit: ${{ github.sha }}"
echo "Author: ${{ github.actor }}"
cat > /tmp/deployment-record.json <<EOF
{
"change_id": "$CHANGE_ID",
"timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
"commit": "${{ github.sha }}",
"author": "${{ github.actor }}",
"environment": "production",
"status": "deployed"
}
EOF
```
## Change Freeze Policy
```yaml
change_freeze:
definition: "Period during which non-emergency changes are prohibited"
scheduled_freezes:
year_end:
start: "December 15"
end: "January 3"
scope: "All production changes"
major_events:
- "Black Friday through Cyber Monday (e-commerce)"
- "Tax filing deadline periods (financial services)"
- "Open enrollment periods (healthcare)"
exceptions_during_freeze:
allowed:
- Security patches for actively exploited vulnerabilities
- Changes required by regulatory deadline
- Fixes for P1/SEV1 production incidents
approval: "VP of Engineering + Security Lead"
communication:
announcement: "2 weeks before freeze"
reminder: "1 week and 1 day before freeze"
daily_status: "During freeze period"
lift_notification: "When freeze ends"
```
## Change Management Metrics
```yaml
metrics:
change_success_rate:
description: "Percentage of changes implemented without rollback or incident"
target: ">95%"
formula: "(successful changes / total changes) * 100"
emergency_change_rate:
description: "Percentage of changes classified as emergency"
target: "<5%"
formula: "(emergency changes / total changes) * 100"
rollback_rate:
description: "Percentage of changes that required rollback"
target: "<3%"
mean_time_to_implement:
description: "Average time from approval to implementation"
target: "Varies by type"
cab_approval_time:
description: "Average time from submission to CAB decision"
target: "<5 business days for normal changes"
```
## Change Management Checklist
```yaml
change_management_checklist:
process_setup:
- [ ] Change types defined with classification criteria
- [ ] Approval matrix documented (who approves what)
- [ ] CAB established with regular meeting schedule
- [ ] Emergency change procedure documented
- [ ] Change request template created
- [ ] Change freeze policy defined
tooling:
- [ ] PR template includes change management fields
- [ ] Automated risk classification in CI/CD
- [ ] Branch protection enforces required approvals
- [ ] Deployment records captured automatically
- [ ] Change audit trail preserved (PR history, approvals)
compliance:
- [ ] All production changes have documented approval
- [ ] Rollback plans exist for every change
- [ ] Post-implementation reviews conducted for failures
- [ ] Emergency changes documented retroactively within 48 hours
- [ ] Change metrics reported monthly
- [ ] Audit trail retained for compliance period (1-3 years)
```
## Best Practices
- Clear change categories
- Required approvals by risk
- Rollback procedures documented
- Post-change verification
- Change freeze windows
- Classify changes by risk level to apply proportionate controls without slowing low-risk work
- Automate risk classification based on files changed, services affected, and deployment scope
- Use PR approvals as the native change approval mechanism for code-driven changes
- Require rollback plans for every change and test rollback procedures periodically
- Track emergency changes as a key metric: a high rate indicates systemic process issues
- Implement change freezes during critical business periods to protect stability
- Conduct post-implementation reviews for all failed changes to drive improvement
- Separate duty of implementation from duty of approval (no self-approving changes)
- Capture deployment records automatically in CI/CD rather than relying on manual entry
- Keep the CAB focused on high-risk decisions; do not bottleneck low-risk changes through CAB
compliance/governance/policy-as-code/SKILL.md
+559 -28
View File
@@ -9,61 +9,592 @@ metadata:
# Policy as Code
Automate policy enforcement through code.
Automate policy enforcement through code using OPA/Rego, Kyverno, Checkov, and CI/CD integration to prevent compliance violations before they reach production.
## Open Policy Agent (OPA)
## When to Use
- Enforcing security and compliance policies on infrastructure-as-code changes
- Preventing misconfigured Kubernetes workloads from deploying
- Automating guardrails in CI/CD pipelines for Terraform, CloudFormation, or Helm
- Implementing organizational standards that must be consistently applied
- Replacing manual approval gates with automated policy checks
## Open Policy Agent (OPA) Rego Policies
```rego
# deny_public_buckets.rego
package terraform.s3
# deny_public_s3.rego - Deny S3 buckets with public access
package terraform.aws.s3
deny[msg] {
resource := input.resource.aws_s3_bucket[name]
resource.acl == "public-read"
msg := sprintf("S3 bucket '%s' has public ACL", [name])
import rego.v1
deny contains msg if {
resource := input.resource_changes[_]
resource.type == "aws_s3_bucket"
resource.change.after.acl == "public-read"
msg := sprintf(
"S3 bucket '%s' has public-read ACL. All buckets must be private. [Policy: no-public-s3]",
[resource.address]
)
}
deny contains msg if {
resource := input.resource_changes[_]
resource.type == "aws_s3_bucket"
resource.change.after.acl == "public-read-write"
msg := sprintf(
"S3 bucket '%s' has public-read-write ACL. This is strictly prohibited. [Policy: no-public-s3]",
[resource.address]
)
}
```
## Kyverno (Kubernetes)
```rego
# require_encryption.rego - Require encryption on data stores
package terraform.aws.encryption
import rego.v1
deny contains msg if {
resource := input.resource_changes[_]
resource.type == "aws_db_instance"
not resource.change.after.storage_encrypted
msg := sprintf(
"RDS instance '%s' does not have storage encryption enabled. [Policy: require-rds-encryption]",
[resource.address]
)
}
deny contains msg if {
resource := input.resource_changes[_]
resource.type == "aws_ebs_volume"
not resource.change.after.encrypted
msg := sprintf(
"EBS volume '%s' is not encrypted. [Policy: require-ebs-encryption]",
[resource.address]
)
}
deny contains msg if {
resource := input.resource_changes[_]
resource.type == "aws_s3_bucket"
not has_encryption(resource)
msg := sprintf(
"S3 bucket '%s' does not have default encryption configured. [Policy: require-s3-encryption]",
[resource.address]
)
}
has_encryption(resource) if {
resource.change.after.server_side_encryption_configuration[_]
}
```
```rego
# require_tags.rego - Enforce mandatory tagging
package terraform.aws.tags
import rego.v1
required_tags := {"Environment", "Owner", "CostCenter", "DataClassification"}
deny contains msg if {
resource := input.resource_changes[_]
tags := object.get(resource.change.after, "tags", {})
missing := required_tags - {key | tags[key]}
count(missing) > 0
msg := sprintf(
"Resource '%s' is missing required tags: %v. [Policy: required-tags]",
[resource.address, missing]
)
}
```
```rego
# restrict_regions.rego - Limit resource deployment to approved regions
package terraform.aws.regions
import rego.v1
approved_regions := {"us-east-1", "us-west-2", "eu-west-1"}
deny contains msg if {
resource := input.resource_changes[_]
provider_config := input.configuration.provider_config.aws
region := provider_config.expressions.region.constant_value
not region in approved_regions
msg := sprintf(
"Resource '%s' is in region '%s'. Approved regions: %v. [Policy: approved-regions]",
[resource.address, region, approved_regions]
)
}
```
```bash
# Evaluate OPA policies against Terraform plan
terraform plan -out=tfplan
terraform show -json tfplan > tfplan.json
# Run OPA evaluation
opa eval \
--data policies/ \
--input tfplan.json \
"data.terraform.aws.s3.deny" \
--format pretty
# Use conftest for easier CI integration
conftest test tfplan.json --policy policies/ --output table
```
## Kyverno Kubernetes Policies
```yaml
# require-labels.yaml - Enforce required labels on all pods
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-labels
annotations:
policies.kyverno.io/title: Require Labels
policies.kyverno.io/category: Best Practices
policies.kyverno.io/severity: medium
spec:
validationFailureAction: enforce
validationFailureAction: Enforce
background: true
rules:
- name: check-labels
- name: check-required-labels
match:
resources:
kinds:
- Pod
any:
- resources:
kinds:
- Pod
validate:
message: "Label 'app' is required"
message: >-
Labels 'app.kubernetes.io/name', 'app.kubernetes.io/version',
and 'team' are required on all Pods.
pattern:
metadata:
labels:
app: "?*"
app.kubernetes.io/name: "?*"
app.kubernetes.io/version: "?*"
team: "?*"
---
# disallow-privileged.yaml - Block privileged containers
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: disallow-privileged-containers
annotations:
policies.kyverno.io/title: Disallow Privileged Containers
policies.kyverno.io/category: Pod Security
policies.kyverno.io/severity: high
spec:
validationFailureAction: Enforce
background: true
rules:
- name: deny-privileged
match:
any:
- resources:
kinds:
- Pod
validate:
message: "Privileged containers are not allowed."
pattern:
spec:
containers:
- securityContext:
privileged: "false"
=(initContainers):
- securityContext:
privileged: "false"
---
# require-resource-limits.yaml - Enforce resource limits
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-resource-limits
annotations:
policies.kyverno.io/title: Require Resource Limits
policies.kyverno.io/severity: medium
spec:
validationFailureAction: Enforce
background: true
rules:
- name: check-resource-limits
match:
any:
- resources:
kinds:
- Pod
validate:
message: "All containers must have CPU and memory limits defined."
pattern:
spec:
containers:
- resources:
limits:
memory: "?*"
cpu: "?*"
---
# disallow-latest-tag.yaml - Block usage of 'latest' image tag
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: disallow-latest-tag
annotations:
policies.kyverno.io/title: Disallow Latest Tag
policies.kyverno.io/severity: medium
spec:
validationFailureAction: Enforce
background: true
rules:
- name: validate-image-tag
match:
any:
- resources:
kinds:
- Pod
validate:
message: "Images must use a specific tag, not 'latest'."
pattern:
spec:
containers:
- image: "!*:latest & *:*"
---
# restrict-image-registries.yaml - Allow only approved registries
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: restrict-image-registries
annotations:
policies.kyverno.io/title: Restrict Image Registries
policies.kyverno.io/severity: high
spec:
validationFailureAction: Enforce
background: true
rules:
- name: validate-registries
match:
any:
- resources:
kinds:
- Pod
validate:
message: >-
Images must come from approved registries:
123456789012.dkr.ecr.us-east-1.amazonaws.com or ghcr.io/your-org.
pattern:
spec:
containers:
- image: "123456789012.dkr.ecr.*.amazonaws.com/* | ghcr.io/your-org/*"
---
# require-networkpolicy.yaml - Ensure namespaces have NetworkPolicies
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-networkpolicy
annotations:
policies.kyverno.io/title: Require Network Policy
policies.kyverno.io/severity: high
spec:
validationFailureAction: Audit
background: true
rules:
- name: check-networkpolicy
match:
any:
- resources:
kinds:
- Deployment
preconditions:
all:
- key: "{{request.object.metadata.namespace}}"
operator: NotIn
value: ["kube-system", "kube-public"]
validate:
message: "A NetworkPolicy must exist in namespace '{{request.object.metadata.namespace}}' before deploying workloads."
deny:
conditions:
all:
- key: "{{request.object.metadata.namespace}}"
operator: AnyNotIn
value: "{{request.object.metadata.namespace}}"
```
## Checkov
## Checkov Custom Checks
```python
# custom_checks/require_s3_versioning.py
from checkov.terraform.checks.resource.base_resource_check import BaseResourceCheck
from checkov.common.models.enums import CheckResult, CheckCategories
class S3Versioning(BaseResourceCheck):
def __init__(self):
name = "Ensure S3 bucket has versioning enabled"
id = "CUSTOM_S3_001"
supported_resources = ["aws_s3_bucket_versioning"]
categories = [CheckCategories.BACKUP_AND_RECOVERY]
super().__init__(name=name, id=id,
categories=categories,
supported_resources=supported_resources)
def scan_resource_conf(self, conf):
versioning = conf.get("versioning_configuration", [{}])
if isinstance(versioning, list):
versioning = versioning[0] if versioning else {}
status = versioning.get("status", ["Disabled"])
if isinstance(status, list):
status = status[0]
return CheckResult.PASSED if status == "Enabled" else CheckResult.FAILED
check = S3Versioning()
```
```python
# custom_checks/require_rds_backup.py
from checkov.terraform.checks.resource.base_resource_check import BaseResourceCheck
from checkov.common.models.enums import CheckResult, CheckCategories
class RDSBackupRetention(BaseResourceCheck):
def __init__(self):
name = "Ensure RDS has backup retention of at least 7 days"
id = "CUSTOM_RDS_001"
supported_resources = ["aws_db_instance"]
categories = [CheckCategories.BACKUP_AND_RECOVERY]
super().__init__(name=name, id=id,
categories=categories,
supported_resources=supported_resources)
def scan_resource_conf(self, conf):
retention = conf.get("backup_retention_period", [0])
if isinstance(retention, list):
retention = retention[0]
return CheckResult.PASSED if int(retention) >= 7 else CheckResult.FAILED
check = RDSBackupRetention()
```
```bash
# Scan Terraform
checkov -d . --framework terraform
# Run Checkov with custom checks
checkov -d ./terraform \
--framework terraform \
--external-checks-dir ./custom_checks \
--output cli \
--compact
# Custom check
from checkov.terraform.checks.resource.base_resource_check import BaseResourceCheck
# Run specific check IDs
checkov -d ./terraform \
--check CUSTOM_S3_001,CUSTOM_RDS_001,CKV_AWS_18,CKV_AWS_19
class S3Encryption(BaseResourceCheck):
def scan_resource_conf(self, conf):
return CheckResult.PASSED if 'encryption' in conf else CheckResult.FAILED
# Generate SARIF output for GitHub Advanced Security integration
checkov -d ./terraform \
--framework terraform \
--output sarif \
--output-file checkov-results.sarif
# Skip specific checks with documented justification
checkov -d ./terraform \
--skip-check CKV_AWS_145 \
--skip-check CKV_AWS_79
```
## CI/CD Pipeline Integration
```yaml
# GitHub Actions - Policy enforcement in PR workflow
name: Policy Checks
on:
pull_request:
paths:
- 'terraform/**'
- 'kubernetes/**'
jobs:
terraform-policy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
- name: Terraform Init and Plan
working-directory: terraform/
run: |
terraform init -backend=false
terraform plan -out=tfplan
terraform show -json tfplan > tfplan.json
- name: OPA Policy Check
uses: open-policy-agent/setup-opa@v2
with:
version: latest
- run: |
RESULTS=$(opa eval \
--data policies/ \
--input terraform/tfplan.json \
--format json \
"data.terraform" | jq '.result[0].expressions[0].value')
DENY_COUNT=$(echo "$RESULTS" | jq '[.. | .deny? // empty | .[] ] | length')
if [ "$DENY_COUNT" -gt 0 ]; then
echo "::error::Policy violations found:"
echo "$RESULTS" | jq '.. | .deny? // empty | .[]'
exit 1
fi
- name: Checkov Scan
uses: bridgecrewio/checkov-action@v12
with:
directory: terraform/
framework: terraform
output_format: cli,sarif
output_file_path: console,checkov-results.sarif
soft_fail: false
external_checks_dirs: custom_checks/
- name: Upload SARIF
if: always()
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: checkov-results.sarif
kubernetes-policy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Kyverno CLI
run: |
curl -LO https://github.com/kyverno/kyverno/releases/latest/download/kyverno-cli_linux_amd64.tar.gz
tar -xzf kyverno-cli_linux_amd64.tar.gz
sudo mv kyverno /usr/local/bin/
- name: Test Kyverno Policies
run: |
kyverno apply policies/kyverno/ \
--resource kubernetes/manifests/ \
--detailed-results \
--output-format table
- name: Conftest Kubernetes Manifests
uses: open-policy-agent/conftest-action@v2
with:
files: kubernetes/manifests/
policy: policies/kubernetes/
```
## Policy Exception Management
```yaml
exception_workflow:
request:
fields:
- policy_id: "Which policy needs an exception"
- resource: "Specific resource requiring exception"
- justification: "Business reason for the exception"
- compensating_controls: "Alternative mitigations in place"
- duration: "Temporary (with expiry) or permanent"
- requestor: "Person requesting"
- approver: "Security team member who approved"
approval_process:
1: "Requestor submits exception with justification"
2: "Security team reviews and assesses risk"
3: "Compensating controls verified"
4: "Exception approved or denied with rationale"
5: "Exception documented in registry"
6: "Automated enforcement updated to allow exception"
enforcement:
opa: |
# Exception list loaded as data
# policies/exceptions.json
# {"exceptions": [{"resource": "aws_s3_bucket.public_website", "policy": "no-public-s3", "expires": "2025-06-01"}]}
kyverno: |
# Use Kyverno PolicyException resource
apiVersion: kyverno.io/v2beta1
kind: PolicyException
metadata:
name: allow-public-website
namespace: web
spec:
exceptions:
- policyName: disallow-privileged-containers
ruleNames:
- deny-privileged
match:
any:
- resources:
kinds:
- Pod
names:
- legacy-app-*
review_schedule:
- Review all active exceptions quarterly
- Expire temporary exceptions automatically
- Re-justify permanent exceptions annually
- Track exception count trends as a security metric
```
## Policy Testing
```bash
# Test OPA policies with mock input
mkdir -p policies/tests
# Create test input
cat > policies/tests/public_bucket_test.json <<'EOF'
{
"resource_changes": [{
"address": "aws_s3_bucket.test",
"type": "aws_s3_bucket",
"change": {
"after": {"acl": "public-read"}
}
}]
}
EOF
# Run test
opa eval --data policies/ --input policies/tests/public_bucket_test.json \
"data.terraform.aws.s3.deny" --format pretty
# Should output the deny message
# OPA unit tests
cat > policies/tests/s3_test.rego <<'EOF'
package terraform.aws.s3_test
import rego.v1
import data.terraform.aws.s3
test_deny_public_bucket if {
result := s3.deny with input as {"resource_changes": [{"address": "test", "type": "aws_s3_bucket", "change": {"after": {"acl": "public-read"}}}]}
count(result) > 0
}
test_allow_private_bucket if {
result := s3.deny with input as {"resource_changes": [{"address": "test", "type": "aws_s3_bucket", "change": {"after": {"acl": "private"}}}]}
count(result) == 0
}
EOF
opa test policies/ -v
```
## Best Practices
- Version control policies
- Test policies in CI
- Gradual rollout (warn → enforce)
- Exception management
- Version control all policies alongside the infrastructure code they govern
- Start in audit/warn mode and transition to enforce after verifying no false positives
- Write unit tests for every policy to catch regressions and verify intended behavior
- Implement a formal exception process: never disable policies to bypass legitimate checks
- Use policy results as PR status checks to block non-compliant merges
- Layer policies: Checkov for static analysis, OPA for Terraform plan evaluation, Kyverno for runtime
- Tag policies with compliance framework references (e.g., SOC 2 CC6.1, PCI Req 2.2)
- Monitor policy violation trends over time to identify systemic issues
- Provide clear, actionable error messages that explain how to fix violations
- Roll out new policies gradually: inform teams, give a remediation window, then enforce
compliance/governance/vendor-management/SKILL.md
+489 -50
View File
@@ -9,65 +9,504 @@ metadata:
# Vendor Management
Manage third-party vendor security risks.
Implement a vendor risk management program covering vendor assessment questionnaires, risk scoring, contract tracking, SLA monitoring, and ongoing oversight for compliance with SOC 2, ISO 27001, and regulatory frameworks.
## Vendor Assessment
## When to Use
- Onboarding new vendors that will access company data or systems
- Conducting annual vendor risk assessments and reassessments
- Negotiating security requirements in vendor contracts
- Monitoring vendor SLA compliance and security posture
- Preparing vendor management evidence for SOC 2 or ISO 27001 audits
## Vendor Risk Tiering
```yaml
assessment_process:
1_identify:
- Catalog all vendors
- Classify by risk tier
2_assess:
- Security questionnaire
- SOC 2 review
- Penetration test results
3_contract:
- Security requirements
- Data processing agreement
- SLAs
4_monitor:
- Continuous monitoring
- Annual reassessment
- Incident notification
vendor_risk_tiers:
critical:
criteria:
- Processes or stores sensitive/regulated data (PII, PHI, PCI)
- Single point of failure (no alternative vendor)
- Has privileged access to production systems
- Handles authentication or security-critical functions
assessment_requirements:
- Full security questionnaire (SIG or custom)
- SOC 2 Type II report review (or equivalent)
- Penetration test results review
- On-site or virtual security assessment (optional)
- Business continuity and DR plan review
review_frequency: Annual
contract_requirements:
- Data processing agreement (DPA)
- Business associate agreement (BAA) if PHI
- Security SLA with breach notification timeline
- Right to audit clause
- Cyber insurance requirements
examples:
- Cloud infrastructure providers (AWS, Azure, GCP)
- Identity providers (Okta, Azure AD)
- Payment processors (Stripe, Adyen)
- Primary database or CRM SaaS
high:
criteria:
- Accesses significant company data (internal or confidential)
- Integrates with production systems via API
- Processes customer-facing transactions
- Substitution would cause significant business disruption
assessment_requirements:
- Security questionnaire
- SOC 2 report review (Type I or Type II)
- Compliance certifications verified
review_frequency: Annual
contract_requirements:
- Data processing agreement
- Security requirements appendix
- Incident notification clause (72 hours)
examples:
- Email/marketing platforms (SendGrid, HubSpot)
- Monitoring and logging SaaS (Datadog, Splunk)
- CI/CD platforms (GitHub, GitLab)
- Customer support platforms
medium:
criteria:
- Limited data access (internal data only)
- Non-production system integration
- Some business impact if unavailable
assessment_requirements:
- Abbreviated security questionnaire
- Compliance certification verification
review_frequency: Every 2 years
contract_requirements:
- Standard vendor terms with security clause
- NDA
examples:
- Project management tools
- HR platforms
- Travel and expense systems
low:
criteria:
- No access to company data
- No system integration
- Easily replaceable
assessment_requirements:
- Basic due diligence (public info review)
- Confirm no data sharing
review_frequency: Every 3 years or on renewal
contract_requirements:
- Standard terms
examples:
- Office supply vendors
- Facilities services
- General consulting (no data access)
```
## Risk Tiers
| Tier | Criteria | Assessment |
|------|----------|------------|
| Critical | Access to sensitive data | Full assessment, annual |
| High | Significant data access | Questionnaire + SOC 2 |
| Medium | Limited data access | Security questionnaire |
| Low | No data access | Basic due diligence |
## Security Questionnaire
## Vendor Assessment Questionnaire
```yaml
categories:
security_questionnaire:
section_1_governance:
questions:
- "Do you have a documented information security policy?"
- "Is there a designated CISO or security lead?"
- "Do you conduct annual security risk assessments?"
- "Do you have a security awareness training program?"
- "What compliance certifications do you hold? (SOC 2, ISO 27001, etc.)"
- "When was your last external security audit?"
- "Do you carry cyber liability insurance? What coverage limits?"
evidence_requested:
- Information security policy (or summary)
- SOC 2 Type II report (or bridge letter)
- ISO 27001 certificate
- Cyber insurance certificate
section_2_access_control:
questions:
- "How do you manage user access to systems containing our data?"
- "Is multi-factor authentication enforced for all personnel?"
- "How frequently do you conduct access reviews?"
- "What is your process for revoking access upon employee termination?"
- "Do you support SSO/SAML integration for customer access?"
- "How do you manage privileged access?"
evidence_requested:
- Access management policy
- MFA configuration documentation
- Access review records (sample)
section_3_data_protection:
questions:
- "How is our data encrypted at rest?"
- "How is our data encrypted in transit?"
- "In which geographic regions is our data stored?"
- "Do you use sub-processors? If so, provide a list."
- "What is your data retention policy?"
- "How is our data isolated from other customers? (multi-tenancy model)"
- "Can you provide data export in standard formats upon request?"
- "What is your data destruction process at contract end?"
evidence_requested:
- Encryption standards documentation
- Sub-processor list
- Data flow diagram showing customer data handling
section_4_vulnerability_management:
questions:
- "How frequently do you perform vulnerability scans?"
- "How frequently do you conduct penetration tests?"
- "What is your patch management SLA for critical vulnerabilities?"
- "Do you have a responsible disclosure or bug bounty program?"
- "How do you manage vulnerabilities in third-party dependencies?"
evidence_requested:
- Penetration test executive summary (last 12 months)
- Vulnerability management policy
- Patch management SLA documentation
section_5_incident_response:
questions:
- "Do you have a documented incident response plan?"
- "What is your breach notification timeline?"
- "Have you experienced a data breach in the last 3 years?"
- "How would you notify us in the event of a security incident?"
- "Do you conduct incident response tabletop exercises?"
evidence_requested:
- Incident response plan summary
- Breach notification procedure
section_6_business_continuity:
questions:
- "Do you have a business continuity plan?"
- "Do you have a disaster recovery plan?"
- "What are your RTO and RPO targets?"
- "How frequently do you test your DR plan?"
- "What is your uptime SLA?"
- "Do you have geographic redundancy?"
evidence_requested:
- BCP/DR plan summary
- Uptime SLA documentation
- Most recent DR test results
section_7_compliance:
questions:
- "Do you process data subject to GDPR, HIPAA, or PCI DSS?"
- "How do you support our compliance obligations?"
- "Do you have a Data Processing Agreement (DPA) template?"
- "How do you handle data subject access requests (DSARs)?"
- "Are you FedRAMP authorized? If so, at what impact level?"
evidence_requested:
- DPA template
- Compliance certification documentation
```
## Risk Scoring Model
```yaml
risk_scoring:
dimensions:
data_sensitivity:
weight: 30
scores:
1: "No access to company or customer data"
2: "Access to public or non-sensitive internal data"
3: "Access to internal confidential data"
4: "Access to PII or customer financial data"
5: "Access to regulated data (PHI, PCI, classified)"
system_access:
weight: 25
scores:
1: "No system access"
2: "Read-only access to non-production"
3: "Read/write access to non-production or read-only production"
4: "Read/write access to production systems"
5: "Privileged/admin access to production or security systems"
business_criticality:
weight: 20
scores:
1: "No operational dependency"
2: "Minor convenience; easily replaced"
3: "Moderate dependency; replacement in weeks"
4: "Significant dependency; replacement in months"
5: "Critical dependency; no viable alternative"
security_posture:
weight: 15
scores:
5: "No certifications, no formal security program"
4: "Some security controls but no external validation"
3: "SOC 2 Type I or equivalent"
2: "SOC 2 Type II within last 12 months"
1: "Multiple certifications (SOC 2 + ISO 27001), strong program"
regulatory_exposure:
weight: 10
scores:
1: "No regulatory requirements"
2: "General data protection (GDPR basic)"
3: "Industry-specific (HIPAA, PCI)"
4: "Government (FedRAMP, ITAR)"
5: "Multiple stringent regulations"
calculation:
formula: "Sum of (dimension_score * dimension_weight) / 100"
risk_levels:
low: "Score 1.0 - 2.0"
medium: "Score 2.1 - 3.0"
high: "Score 3.1 - 4.0"
critical: "Score 4.1 - 5.0"
example:
vendor: "Payment Processor X"
data_sensitivity: 5 # PCI data
system_access: 4 # Production API integration
business_criticality: 5 # No alternative
security_posture: 2 # SOC 2 Type II
regulatory_exposure: 3 # PCI DSS
score: "(5*30 + 4*25 + 5*20 + 2*15 + 3*10) / 100 = 4.1 -> Critical"
```
## Vendor Registry and Contract Tracking
```yaml
vendor_registry_schema:
vendor_info:
vendor_id: "VND-NNNN"
vendor_name: ""
vendor_website: ""
primary_contact_email: ""
security_contact_email: ""
vendor_category: "" # SaaS, IaaS, Consulting, etc.
risk_assessment:
risk_tier: "" # critical, high, medium, low
risk_score: 0.0
last_assessment_date: ""
next_assessment_date: ""
assessment_status: "" # current, due, overdue
open_findings: 0
certifications:
- type: "SOC 2 Type II"
valid_until: ""
report_on_file: true
- type: "ISO 27001"
valid_until: ""
certificate_on_file: true
contract:
contract_id: ""
start_date: ""
end_date: ""
auto_renewal: true
cancellation_notice_days: 90
annual_value: 0
terms:
data_processing_agreement: true
nda: true
baa: false
right_to_audit: true
breach_notification_sla: "72 hours"
data_return_clause: true
data_destruction_clause: true
cyber_insurance_required: true
data_access:
data_types: []
data_classification: ""
data_location: []
sub_processors: []
sla_tracking:
uptime_sla: "99.9%"
actual_uptime_last_month: ""
support_response_sla: ""
sla_breaches_ytd: 0
status: "" # active, under_review, offboarding, inactive
owner: "" # Internal team/person responsible
```
## SLA Monitoring
```python
"""
Vendor SLA monitoring - Track uptime and response time commitments.
"""
import requests
from datetime import datetime, timezone
class VendorSLAMonitor:
def __init__(self, vendors_config):
self.vendors = vendors_config
def check_uptime(self, vendor):
"""Check vendor service availability."""
results = []
for endpoint in vendor.get("health_endpoints", []):
try:
resp = requests.get(
endpoint["url"],
timeout=endpoint.get("timeout", 10),
headers=endpoint.get("headers", {}),
)
results.append({
"endpoint": endpoint["url"],
"status": resp.status_code,
"response_time_ms": resp.elapsed.total_seconds() * 1000,
"healthy": resp.status_code == endpoint.get("expected_status", 200),
"timestamp": datetime.now(timezone.utc).isoformat(),
})
except requests.RequestException as e:
results.append({
"endpoint": endpoint["url"],
"status": "error",
"error": str(e),
"healthy": False,
"timestamp": datetime.now(timezone.utc).isoformat(),
})
return results
def check_status_page(self, vendor):
"""Check vendor status page for active incidents."""
status_url = vendor.get("status_page_url")
if not status_url:
return None
try:
api_url = f"{status_url}/api/v2/summary.json"
resp = requests.get(api_url, timeout=10)
data = resp.json()
return {
"vendor": vendor["name"],
"status": data.get("status", {}).get("indicator", "unknown"),
"active_incidents": len(data.get("incidents", [])),
"components": [
{"name": c["name"], "status": c["status"]}
for c in data.get("components", [])
],
}
except Exception:
return {"vendor": vendor["name"], "status": "unknown"}
def generate_sla_report(self, vendor_name, monthly_checks):
"""Calculate monthly SLA compliance."""
total = len(monthly_checks)
healthy = sum(1 for c in monthly_checks if c.get("healthy"))
uptime_pct = (healthy / total * 100) if total > 0 else 0
avg_response = (
sum(c.get("response_time_ms", 0) for c in monthly_checks if c.get("healthy"))
/ max(healthy, 1)
)
return {
"vendor": vendor_name,
"period": datetime.now(timezone.utc).strftime("%Y-%m"),
"total_checks": total,
"healthy_checks": healthy,
"uptime_percentage": round(uptime_pct, 3),
"avg_response_time_ms": round(avg_response, 1),
"sla_met": uptime_pct >= 99.9,
}
```
## Vendor Lifecycle Management
```yaml
vendor_lifecycle:
onboarding:
step_1_request:
- Business owner submits vendor request with use case
- Procurement assigns vendor ID
- Initial risk tier assessment based on data access and criticality
step_2_assess:
- Send security questionnaire (appropriate to tier)
- Review compliance certifications
- Evaluate questionnaire responses
- Score vendor risk
step_3_contract:
- Negotiate security requirements based on risk tier
- Execute DPA/BAA as required
- Document data flows and access scope
- Set SLA expectations
step_4_provision:
- Configure integration with least privilege access
- Enable audit logging for vendor access
- Add to vendor registry
- Schedule first reassessment
ongoing_management:
monitoring:
- Track SLA compliance monthly
- Monitor vendor status pages for incidents
- Review vendor security advisories
- Track data sub-processor changes
reassessment:
- Conduct reassessment per tier schedule
- Review updated SOC 2 / ISO 27001 reports
- Verify certifications are current
- Update risk score
offboarding:
step_1_plan:
- Data migration or transition to replacement vendor
- Identify all integrations and access points
- Communication plan for stakeholders
step_2_execute:
- Revoke all API keys, credentials, and access
- Request data return or destruction certificate
- Remove vendor integrations from systems
- Disable SSO/SAML connections
step_3_verify:
- Confirm data destruction (written certification)
- Verify all access revoked
- Update vendor registry status to inactive
- Archive vendor records for retention period
```
## Vendor Management Checklist
```yaml
vendor_management_checklist:
program_setup:
- [ ] Vendor risk tiering criteria defined
- [ ] Security questionnaire template created
- [ ] Risk scoring model documented
- [ ] Vendor registry established
- [ ] Onboarding and offboarding procedures documented
- [ ] Contract security requirements defined per tier
ongoing_operations:
- [ ] All active vendors cataloged in registry
- [ ] Risk tier assigned to each vendor
- [ ] Security assessments current (per tier schedule)
- [ ] Compliance certifications on file and not expired
- [ ] DPAs/BAAs signed for all vendors handling personal data
- [ ] SLA monitoring active for critical and high-tier vendors
- [ ] Sub-processor lists reviewed and tracked
- [ ] Vendor security incidents tracked and assessed
governance:
- Security policies
- Risk management
- Compliance certifications
technical:
- Access controls
- Encryption
- Vulnerability management
operational:
- Incident response
- Business continuity
- Change management
- [ ] Vendor management policy approved and published
- [ ] Roles and responsibilities assigned (owner per vendor)
- [ ] Assessment findings tracked to remediation
- [ ] Vendor risk reported to management quarterly
- [ ] Offboarding includes data destruction verification
- [ ] Evidence retained for compliance audit (3+ years)
```
## Best Practices
- Tier-based assessments
- Regular reassessment
- Contract security terms
- Incident notification requirements
- Exit strategy planning
- Tier vendors by risk before investing assessment effort: not every vendor needs a full security review
- Use standardized questionnaires (SIG, CAIQ, or consistent custom template) for comparable assessments
- Review SOC 2 Type II reports thoroughly, including complementary user entity controls
- Include right-to-audit clauses in contracts for critical vendors even if you do not exercise them frequently
- Monitor vendor status pages and set up alerts for outages affecting your services
- Track sub-processor changes: your vendor's vendor is part of your supply chain risk
- Maintain a vendor registry as a single source of truth for all vendor relationships
- Conduct offboarding rigorously: revoke all access and obtain data destruction certificates
- Score vendor risk quantitatively to enable consistent prioritization and trend analysis
- Report vendor risk metrics to management quarterly as part of the overall risk management program