mirror of
https://github.com/BagelHole/DevOps-Security-Agent-Skills.git
synced 2026-08-22 12:49:53 +02:00
V2
This commit is contained in:
Toby
@@ -14,72 +14,449 @@ Securely store, manage, and rotate secrets in AWS.
|
||||
## When to Use This Skill
|
||||
|
||||
Use this skill when:
|
||||
- Storing database credentials
|
||||
- Managing API keys in AWS
|
||||
- Implementing automatic secret rotation
|
||||
- Integrating secrets with AWS services
|
||||
- Storing database credentials, API keys, or tokens in AWS
|
||||
- Implementing automatic credential rotation for RDS or other services
|
||||
- Replacing hardcoded secrets in application code or config files
|
||||
- Integrating secrets into ECS, EKS, or Lambda workloads
|
||||
- Meeting compliance requirements for secret management and rotation
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- AWS account
|
||||
- AWS CLI configured
|
||||
- IAM permissions for Secrets Manager
|
||||
- AWS account with appropriate IAM permissions
|
||||
- AWS CLI v2 installed and configured
|
||||
- IAM policy allowing `secretsmanager:*` actions (or scoped permissions)
|
||||
- For rotation: Lambda execution role and VPC access to target services
|
||||
- Python 3.9+ with `boto3` for SDK examples
|
||||
|
||||
## Basic Operations
|
||||
## Secret Creation and Management
|
||||
|
||||
```bash
|
||||
# Create secret
|
||||
# Create a secret with JSON structure
|
||||
aws secretsmanager create-secret \
|
||||
--name myapp/database \
|
||||
--secret-string '{"username":"admin","password":"secret123"}'
|
||||
--name myapp/production/database \
|
||||
--description "Production database credentials" \
|
||||
--secret-string '{"username":"dbadmin","password":"S3cur3P@ssw0rd!","engine":"postgres","host":"db.internal.example.com","port":5432,"dbname":"myapp"}' \
|
||||
--tags '[{"Key":"Environment","Value":"production"},{"Key":"Team","Value":"platform"}]'
|
||||
|
||||
# Get secret
|
||||
aws secretsmanager get-secret-value --secret-id myapp/database
|
||||
# Create a secret with KMS encryption (custom key)
|
||||
aws secretsmanager create-secret \
|
||||
--name myapp/production/api-key \
|
||||
--description "Third-party API key" \
|
||||
--secret-string "ak_live_xxxxxxxxxxxx" \
|
||||
--kms-key-id alias/secrets-key
|
||||
|
||||
# Update secret
|
||||
# Create a binary secret (certificates, keys)
|
||||
aws secretsmanager create-secret \
|
||||
--name myapp/production/tls-cert \
|
||||
--secret-binary fileb://server.pfx
|
||||
|
||||
# Get secret value
|
||||
aws secretsmanager get-secret-value \
|
||||
--secret-id myapp/production/database \
|
||||
--query 'SecretString' --output text | jq .
|
||||
|
||||
# Get a specific version
|
||||
aws secretsmanager get-secret-value \
|
||||
--secret-id myapp/production/database \
|
||||
--version-stage AWSPREVIOUS
|
||||
|
||||
# Update secret value
|
||||
aws secretsmanager put-secret-value \
|
||||
--secret-id myapp/database \
|
||||
--secret-string '{"username":"admin","password":"newpassword"}'
|
||||
--secret-id myapp/production/database \
|
||||
--secret-string '{"username":"dbadmin","password":"N3wS3cur3P@ss!","engine":"postgres","host":"db.internal.example.com","port":5432,"dbname":"myapp"}'
|
||||
|
||||
# Delete secret
|
||||
aws secretsmanager delete-secret --secret-id myapp/database --recovery-window-in-days 7
|
||||
# List all secrets
|
||||
aws secretsmanager list-secrets \
|
||||
--filters Key=name,Values=myapp/production
|
||||
|
||||
# Delete secret (with recovery window)
|
||||
aws secretsmanager delete-secret \
|
||||
--secret-id myapp/production/old-key \
|
||||
--recovery-window-in-days 7
|
||||
|
||||
# Restore a deleted secret
|
||||
aws secretsmanager restore-secret \
|
||||
--secret-id myapp/production/old-key
|
||||
|
||||
# Tag a secret
|
||||
aws secretsmanager tag-resource \
|
||||
--secret-id myapp/production/database \
|
||||
--tags '[{"Key":"RotationEnabled","Value":"true"}]'
|
||||
```
|
||||
|
||||
## Automatic Rotation
|
||||
|
||||
### Enable Rotation
|
||||
|
||||
```bash
|
||||
# Enable rotation with Lambda
|
||||
# Enable rotation with an existing Lambda function
|
||||
aws secretsmanager rotate-secret \
|
||||
--secret-id myapp/database \
|
||||
--rotation-lambda-arn arn:aws:lambda:region:account:function:rotation-function \
|
||||
--rotation-rules AutomaticallyAfterDays=30
|
||||
--secret-id myapp/production/database \
|
||||
--rotation-lambda-arn arn:aws:lambda:us-east-1:123456789012:function:SecretsManagerRDSPostgreSQLRotation \
|
||||
--rotation-rules '{"AutomaticallyAfterDays":30,"ScheduleExpression":"rate(30 days)"}'
|
||||
|
||||
# Trigger immediate rotation
|
||||
aws secretsmanager rotate-secret \
|
||||
--secret-id myapp/production/database
|
||||
|
||||
# Check rotation status
|
||||
aws secretsmanager describe-secret \
|
||||
--secret-id myapp/production/database \
|
||||
--query '{RotationEnabled:RotationEnabled,RotationLambdaARN:RotationLambdaARN,RotationRules:RotationRules,LastRotatedDate:LastRotatedDate}'
|
||||
```
|
||||
|
||||
### Lambda Rotation Function
|
||||
|
||||
```python
|
||||
"""rotation_function.py - Custom rotation Lambda for database credentials."""
|
||||
|
||||
import boto3
|
||||
import json
|
||||
import logging
|
||||
import psycopg2
|
||||
|
||||
logger = logging.getLogger()
|
||||
logger.setLevel(logging.INFO)
|
||||
|
||||
def lambda_handler(event, context):
|
||||
"""Secrets Manager rotation handler.
|
||||
|
||||
The rotation process has four steps:
|
||||
1. createSecret - Generate new secret value
|
||||
2. setSecret - Apply the new secret to the target service
|
||||
3. testSecret - Verify the new secret works
|
||||
4. finishSecret - Mark rotation complete
|
||||
"""
|
||||
secret_arn = event['SecretId']
|
||||
token = event['ClientRequestToken']
|
||||
step = event['Step']
|
||||
|
||||
client = boto3.client('secretsmanager')
|
||||
|
||||
metadata = client.describe_secret(SecretId=secret_arn)
|
||||
if not metadata.get('RotationEnabled'):
|
||||
raise ValueError(f"Secret {secret_arn} does not have rotation enabled")
|
||||
|
||||
versions = metadata.get('VersionIdsToStages', {})
|
||||
if token not in versions:
|
||||
raise ValueError(f"Secret version {token} has no stage for rotation")
|
||||
|
||||
if step == "createSecret":
|
||||
create_secret(client, secret_arn, token)
|
||||
elif step == "setSecret":
|
||||
set_secret(client, secret_arn, token)
|
||||
elif step == "testSecret":
|
||||
test_secret(client, secret_arn, token)
|
||||
elif step == "finishSecret":
|
||||
finish_secret(client, secret_arn, token)
|
||||
else:
|
||||
raise ValueError(f"Invalid step: {step}")
|
||||
|
||||
|
||||
def create_secret(client, secret_arn, token):
|
||||
"""Generate a new secret value."""
|
||||
current = client.get_secret_value(
|
||||
SecretId=secret_arn, VersionStage="AWSCURRENT"
|
||||
)
|
||||
current_dict = json.loads(current['SecretString'])
|
||||
|
||||
new_password = client.get_random_password(
|
||||
PasswordLength=32,
|
||||
ExcludeCharacters='/@"\\',
|
||||
RequireEachIncludedType=True,
|
||||
)['RandomPassword']
|
||||
|
||||
current_dict['password'] = new_password
|
||||
client.put_secret_value(
|
||||
SecretId=secret_arn,
|
||||
ClientRequestToken=token,
|
||||
SecretString=json.dumps(current_dict),
|
||||
VersionStages=['AWSPENDING'],
|
||||
)
|
||||
logger.info(f"createSecret: New secret version created for {secret_arn}")
|
||||
|
||||
|
||||
def set_secret(client, secret_arn, token):
|
||||
"""Apply the new secret to the target database."""
|
||||
pending = client.get_secret_value(
|
||||
SecretId=secret_arn, VersionId=token, VersionStage="AWSPENDING"
|
||||
)
|
||||
pending_dict = json.loads(pending['SecretString'])
|
||||
|
||||
current = client.get_secret_value(
|
||||
SecretId=secret_arn, VersionStage="AWSCURRENT"
|
||||
)
|
||||
current_dict = json.loads(current['SecretString'])
|
||||
|
||||
conn = psycopg2.connect(
|
||||
host=current_dict['host'],
|
||||
port=current_dict.get('port', 5432),
|
||||
user=current_dict['username'],
|
||||
password=current_dict['password'],
|
||||
dbname=current_dict.get('dbname', 'postgres'),
|
||||
)
|
||||
conn.autocommit = True
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"ALTER USER %s WITH PASSWORD %s",
|
||||
(pending_dict['username'], pending_dict['password']),
|
||||
)
|
||||
conn.close()
|
||||
logger.info(f"setSecret: Password updated in database for {secret_arn}")
|
||||
|
||||
|
||||
def test_secret(client, secret_arn, token):
|
||||
"""Verify the new secret works."""
|
||||
pending = client.get_secret_value(
|
||||
SecretId=secret_arn, VersionId=token, VersionStage="AWSPENDING"
|
||||
)
|
||||
pending_dict = json.loads(pending['SecretString'])
|
||||
|
||||
conn = psycopg2.connect(
|
||||
host=pending_dict['host'],
|
||||
port=pending_dict.get('port', 5432),
|
||||
user=pending_dict['username'],
|
||||
password=pending_dict['password'],
|
||||
dbname=pending_dict.get('dbname', 'postgres'),
|
||||
)
|
||||
conn.close()
|
||||
logger.info(f"testSecret: New credentials verified for {secret_arn}")
|
||||
|
||||
|
||||
def finish_secret(client, secret_arn, token):
|
||||
"""Finalize the rotation by updating version stages."""
|
||||
metadata = client.describe_secret(SecretId=secret_arn)
|
||||
versions = metadata.get('VersionIdsToStages', {})
|
||||
|
||||
current_version = None
|
||||
for version_id, stages in versions.items():
|
||||
if "AWSCURRENT" in stages:
|
||||
if version_id == token:
|
||||
logger.info("finishSecret: Version already marked AWSCURRENT")
|
||||
return
|
||||
current_version = version_id
|
||||
break
|
||||
|
||||
client.update_secret_version_stage(
|
||||
SecretId=secret_arn,
|
||||
VersionStage="AWSCURRENT",
|
||||
MoveToVersionId=token,
|
||||
RemoveFromVersionId=current_version,
|
||||
)
|
||||
logger.info(f"finishSecret: Rotation complete for {secret_arn}")
|
||||
```
|
||||
|
||||
### Rotation Lambda Terraform
|
||||
|
||||
```hcl
|
||||
resource "aws_lambda_function" "rotation" {
|
||||
filename = "rotation_function.zip"
|
||||
function_name = "secrets-rotation-postgresql"
|
||||
role = aws_iam_role.rotation.arn
|
||||
handler = "rotation_function.lambda_handler"
|
||||
runtime = "python3.11"
|
||||
timeout = 60
|
||||
|
||||
vpc_config {
|
||||
subnet_ids = var.private_subnet_ids
|
||||
security_group_ids = [aws_security_group.rotation.id]
|
||||
}
|
||||
|
||||
environment {
|
||||
variables = {
|
||||
SECRETS_MANAGER_ENDPOINT = "https://secretsmanager.${var.region}.amazonaws.com"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "aws_lambda_permission" "secrets_manager" {
|
||||
action = "lambda:InvokeFunction"
|
||||
function_name = aws_lambda_function.rotation.function_name
|
||||
principal = "secretsmanager.amazonaws.com"
|
||||
statement_id = "AllowSecretsManager"
|
||||
}
|
||||
|
||||
resource "aws_secretsmanager_secret_rotation" "db" {
|
||||
secret_id = aws_secretsmanager_secret.db.id
|
||||
rotation_lambda_arn = aws_lambda_function.rotation.arn
|
||||
rotation_rules {
|
||||
automatically_after_days = 30
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Application Integration
|
||||
|
||||
### Python SDK
|
||||
|
||||
```python
|
||||
import boto3
|
||||
import json
|
||||
from functools import lru_cache
|
||||
|
||||
def get_secret(secret_name):
|
||||
client = boto3.client('secretsmanager')
|
||||
def get_secret(secret_name: str, region: str = "us-east-1") -> dict:
|
||||
"""Retrieve and parse a secret from AWS Secrets Manager."""
|
||||
client = boto3.client("secretsmanager", region_name=region)
|
||||
response = client.get_secret_value(SecretId=secret_name)
|
||||
return json.loads(response['SecretString'])
|
||||
if "SecretString" in response:
|
||||
return json.loads(response["SecretString"])
|
||||
else:
|
||||
import base64
|
||||
return base64.b64decode(response["SecretBinary"])
|
||||
|
||||
@lru_cache(maxsize=32)
|
||||
def get_cached_secret(secret_name: str) -> dict:
|
||||
"""Cached secret retrieval. Clear cache on rotation events."""
|
||||
return get_secret(secret_name)
|
||||
|
||||
# Usage
|
||||
creds = get_secret('myapp/database')
|
||||
db_connect(creds['username'], creds['password'])
|
||||
creds = get_secret("myapp/production/database")
|
||||
connection_string = (
|
||||
f"postgresql://{creds['username']}:{creds['password']}"
|
||||
f"@{creds['host']}:{creds['port']}/{creds['dbname']}"
|
||||
)
|
||||
```
|
||||
|
||||
### ECS Task Definition
|
||||
|
||||
```json
|
||||
{
|
||||
"containerDefinitions": [
|
||||
{
|
||||
"name": "myapp",
|
||||
"image": "ghcr.io/acme/myapp:v1.0.0",
|
||||
"secrets": [
|
||||
{
|
||||
"name": "DB_USERNAME",
|
||||
"valueFrom": "arn:aws:secretsmanager:us-east-1:123456789:secret:myapp/production/database:username::"
|
||||
},
|
||||
{
|
||||
"name": "DB_PASSWORD",
|
||||
"valueFrom": "arn:aws:secretsmanager:us-east-1:123456789:secret:myapp/production/database:password::"
|
||||
},
|
||||
{
|
||||
"name": "API_KEY",
|
||||
"valueFrom": "arn:aws:secretsmanager:us-east-1:123456789:secret:myapp/production/api-key"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"executionRoleArn": "arn:aws:iam::123456789:role/ecsTaskExecutionRole"
|
||||
}
|
||||
```
|
||||
|
||||
### EKS with External Secrets Operator
|
||||
|
||||
```yaml
|
||||
apiVersion: external-secrets.io/v1beta1
|
||||
kind: SecretStore
|
||||
metadata:
|
||||
name: aws-secrets-manager
|
||||
namespace: production
|
||||
spec:
|
||||
provider:
|
||||
aws:
|
||||
service: SecretsManager
|
||||
region: us-east-1
|
||||
auth:
|
||||
jwt:
|
||||
serviceAccountRef:
|
||||
name: external-secrets-sa
|
||||
---
|
||||
apiVersion: external-secrets.io/v1beta1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: db-credentials
|
||||
namespace: production
|
||||
spec:
|
||||
refreshInterval: 1h
|
||||
secretStoreRef:
|
||||
name: aws-secrets-manager
|
||||
kind: SecretStore
|
||||
target:
|
||||
name: db-credentials
|
||||
creationPolicy: Owner
|
||||
data:
|
||||
- secretKey: username
|
||||
remoteRef:
|
||||
key: myapp/production/database
|
||||
property: username
|
||||
- secretKey: password
|
||||
remoteRef:
|
||||
key: myapp/production/database
|
||||
property: password
|
||||
```
|
||||
|
||||
## Resource-Based Policy
|
||||
|
||||
```bash
|
||||
# Restrict secret access to specific roles
|
||||
aws secretsmanager put-resource-policy \
|
||||
--secret-id myapp/production/database \
|
||||
--resource-policy '{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": {
|
||||
"AWS": [
|
||||
"arn:aws:iam::123456789:role/myapp-ecs-task-role",
|
||||
"arn:aws:iam::123456789:role/myapp-lambda-role"
|
||||
]
|
||||
},
|
||||
"Action": [
|
||||
"secretsmanager:GetSecretValue"
|
||||
],
|
||||
"Resource": "*",
|
||||
"Condition": {
|
||||
"StringEquals": {
|
||||
"aws:RequestedRegion": "us-east-1"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"Effect": "Deny",
|
||||
"Principal": "*",
|
||||
"Action": "secretsmanager:GetSecretValue",
|
||||
"Resource": "*",
|
||||
"Condition": {
|
||||
"StringNotEquals": {
|
||||
"aws:PrincipalAccount": "123456789012"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| `AccessDeniedException` on GetSecretValue | IAM policy missing permission | Add `secretsmanager:GetSecretValue` to the role; check resource-based policy |
|
||||
| Rotation fails with Lambda timeout | Lambda cannot reach database | Ensure Lambda is in same VPC with route to DB; check security groups |
|
||||
| Secret value is empty after rotation | createSecret step failed | Check Lambda CloudWatch logs; verify random password generation works |
|
||||
| ECS container fails to start | Secret ARN format incorrect | Use full ARN with `::` for JSON key extraction; verify secret exists |
|
||||
| Application uses old credentials after rotation | Client caching stale values | Implement cache invalidation on rotation; reduce cache TTL |
|
||||
| Rotation Lambda permission error | Missing `lambda:InvokeFunction` permission | Add `aws_lambda_permission` for secretsmanager.amazonaws.com principal |
|
||||
| KMS decrypt fails | Secret KMS key policy missing role | Add the accessing role to the KMS key policy's `kms:Decrypt` principals |
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Enable automatic rotation
|
||||
- Use resource-based policies
|
||||
- Enable encryption with KMS
|
||||
- Implement least-privilege access
|
||||
- Use versioning for rollback
|
||||
- Enable automatic rotation with 30-day intervals minimum
|
||||
- Use resource-based policies in addition to IAM policies (defense in depth)
|
||||
- Encrypt secrets with customer-managed KMS keys (not default)
|
||||
- Implement least-privilege access (only the roles that need each secret)
|
||||
- Use secret versioning for safe rollback during rotation issues
|
||||
- Monitor secret access with CloudTrail and alert on unusual patterns
|
||||
- Structure secret names hierarchically: `{app}/{env}/{secret-type}`
|
||||
- Never log secret values; log only secret ARNs and access metadata
|
||||
- Test rotation in staging before enabling in production
|
||||
- Set up CloudWatch alarms for rotation failures
|
||||
|
||||
## Related Skills
|
||||
|
||||
- [hashicorp-vault](../hashicorp-vault/) - Multi-cloud secrets
|
||||
- [aws-iam](../../../infrastructure/cloud-aws/aws-iam/) - IAM policies
|
||||
- [azure-keyvault](../azure-keyvault/) - Azure secret management
|
||||
- [gcp-secret-manager](../gcp-secret-manager/) - GCP secret management
|
||||
|
||||
@@ -14,75 +14,484 @@ Securely store and manage secrets, keys, and certificates in Azure.
|
||||
## When to Use This Skill
|
||||
|
||||
Use this skill when:
|
||||
- Managing secrets in Azure
|
||||
- Storing encryption keys
|
||||
- Managing SSL certificates
|
||||
- Integrating with Azure services
|
||||
- Managing secrets, encryption keys, or certificates in Azure
|
||||
- Implementing centralized secret management for Azure services
|
||||
- Integrating secrets into AKS, App Service, or Azure Functions
|
||||
- Encrypting data with customer-managed keys (CMK)
|
||||
- Meeting compliance requirements for key management (FIPS 140-2)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Azure subscription
|
||||
- Azure CLI installed
|
||||
- Appropriate RBAC permissions
|
||||
- Azure subscription with appropriate permissions
|
||||
- Azure CLI installed (`az` command)
|
||||
- Contributor or Key Vault Administrator role for vault management
|
||||
- Managed identity configured for application access
|
||||
- Understanding of Azure RBAC vs. Key Vault access policies
|
||||
|
||||
## Basic Operations
|
||||
## Vault Creation and Configuration
|
||||
|
||||
```bash
|
||||
# Create Key Vault
|
||||
az keyvault create --name mykeyvault --resource-group mygroup --location eastus
|
||||
# Create a resource group
|
||||
az group create --name rg-secrets --location eastus
|
||||
|
||||
# Set secret
|
||||
az keyvault secret set --vault-name mykeyvault --name db-password --value "secret123"
|
||||
# Create Key Vault with RBAC authorization (recommended)
|
||||
az keyvault create \
|
||||
--name myapp-vault-prod \
|
||||
--resource-group rg-secrets \
|
||||
--location eastus \
|
||||
--enable-rbac-authorization true \
|
||||
--enable-soft-delete true \
|
||||
--retention-days 90 \
|
||||
--enable-purge-protection true \
|
||||
--sku premium # Use premium for HSM-backed keys
|
||||
|
||||
# Get secret
|
||||
az keyvault secret show --vault-name mykeyvault --name db-password
|
||||
# Create Key Vault with access policies (legacy)
|
||||
az keyvault create \
|
||||
--name myapp-vault-dev \
|
||||
--resource-group rg-secrets \
|
||||
--location eastus \
|
||||
--enable-soft-delete true \
|
||||
--retention-days 30
|
||||
|
||||
# List secrets
|
||||
az keyvault secret list --vault-name mykeyvault
|
||||
# Enable private endpoint (no public access)
|
||||
az keyvault update \
|
||||
--name myapp-vault-prod \
|
||||
--resource-group rg-secrets \
|
||||
--public-network-access Disabled
|
||||
|
||||
# Enable diagnostics logging
|
||||
az monitor diagnostic-settings create \
|
||||
--name kv-diagnostics \
|
||||
--resource "/subscriptions/{sub}/resourceGroups/rg-secrets/providers/Microsoft.KeyVault/vaults/myapp-vault-prod" \
|
||||
--workspace "/subscriptions/{sub}/resourceGroups/rg-monitor/providers/Microsoft.OperationalInsights/workspaces/security-logs" \
|
||||
--logs '[{"category":"AuditEvent","enabled":true,"retentionPolicy":{"enabled":true,"days":365}}]'
|
||||
```
|
||||
|
||||
## Secret Management
|
||||
|
||||
```bash
|
||||
# Set a secret
|
||||
az keyvault secret set \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name db-password \
|
||||
--value "S3cur3P@ssw0rd!" \
|
||||
--content-type "text/plain" \
|
||||
--tags Environment=production Team=platform
|
||||
|
||||
# Set a multi-line secret (JSON credentials)
|
||||
az keyvault secret set \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name db-credentials \
|
||||
--value '{"username":"dbadmin","password":"S3cur3P@ss!","host":"db.postgres.database.azure.com","port":5432}'
|
||||
|
||||
# Get secret value
|
||||
az keyvault secret show \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name db-password \
|
||||
--query value -o tsv
|
||||
|
||||
# Get specific version
|
||||
az keyvault secret show \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name db-password \
|
||||
--version abc123def456
|
||||
|
||||
# List all secrets
|
||||
az keyvault secret list --vault-name myapp-vault-prod -o table
|
||||
|
||||
# List secret versions
|
||||
az keyvault secret list-versions \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name db-password -o table
|
||||
|
||||
# Set expiration date
|
||||
az keyvault secret set-attributes \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name api-key \
|
||||
--expires "2026-01-01T00:00:00Z"
|
||||
|
||||
# Disable a secret (without deleting)
|
||||
az keyvault secret set-attributes \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name old-api-key \
|
||||
--enabled false
|
||||
|
||||
# Delete a secret (soft-delete)
|
||||
az keyvault secret delete \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name old-api-key
|
||||
|
||||
# Recover a deleted secret
|
||||
az keyvault secret recover \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name old-api-key
|
||||
|
||||
# Purge a deleted secret (permanent, requires purge protection to be off)
|
||||
az keyvault secret purge \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name old-api-key
|
||||
|
||||
# Backup and restore
|
||||
az keyvault secret backup \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name db-password \
|
||||
--file db-password.backup
|
||||
|
||||
az keyvault secret restore \
|
||||
--vault-name myapp-vault-prod \
|
||||
--file db-password.backup
|
||||
```
|
||||
|
||||
## Key Management
|
||||
|
||||
```bash
|
||||
# Create an RSA key for encryption
|
||||
az keyvault key create \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name data-encryption-key \
|
||||
--kty RSA \
|
||||
--size 4096 \
|
||||
--ops encrypt decrypt wrapKey unwrapKey
|
||||
|
||||
# Create an EC key for signing
|
||||
az keyvault key create \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name signing-key \
|
||||
--kty EC \
|
||||
--curve P-256 \
|
||||
--ops sign verify
|
||||
|
||||
# Import an existing key
|
||||
az keyvault key import \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name imported-key \
|
||||
--pem-file key.pem
|
||||
|
||||
# Encrypt data
|
||||
az keyvault key encrypt \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name data-encryption-key \
|
||||
--algorithm RSA-OAEP-256 \
|
||||
--value "base64-encoded-plaintext"
|
||||
|
||||
# Rotate a key
|
||||
az keyvault key rotate \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name data-encryption-key
|
||||
|
||||
# Set key rotation policy
|
||||
az keyvault key rotation-policy update \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name data-encryption-key \
|
||||
--value '{
|
||||
"lifetimeActions": [
|
||||
{
|
||||
"trigger": {"timeBeforeExpiry": "P30D"},
|
||||
"action": {"type": "Notify"}
|
||||
},
|
||||
{
|
||||
"trigger": {"timeAfterCreate": "P90D"},
|
||||
"action": {"type": "Rotate"}
|
||||
}
|
||||
],
|
||||
"attributes": {"expiryTime": "P180D"}
|
||||
}'
|
||||
```
|
||||
|
||||
## Certificate Management
|
||||
|
||||
```bash
|
||||
# Create a self-signed certificate
|
||||
az keyvault certificate create \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name app-tls-cert \
|
||||
--policy '{
|
||||
"issuerParameters": {"name": "Self"},
|
||||
"keyProperties": {"exportable": true, "keySize": 4096, "keyType": "RSA"},
|
||||
"secretProperties": {"contentType": "application/x-pkcs12"},
|
||||
"x509CertificateProperties": {
|
||||
"subject": "CN=app.example.com",
|
||||
"subjectAlternativeNames": {"dnsNames": ["app.example.com", "*.app.example.com"]},
|
||||
"validityInMonths": 12,
|
||||
"keyUsage": ["digitalSignature", "keyEncipherment"],
|
||||
"ekus": ["1.3.6.1.5.5.7.3.1"]
|
||||
},
|
||||
"lifetimeActions": [
|
||||
{"trigger": {"daysBeforeExpiry": 30}, "action": {"actionType": "AutoRenew"}}
|
||||
]
|
||||
}'
|
||||
|
||||
# Import a certificate
|
||||
az keyvault certificate import \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name imported-cert \
|
||||
--file certificate.pfx \
|
||||
--password "pfx-password"
|
||||
|
||||
# Download certificate
|
||||
az keyvault certificate download \
|
||||
--vault-name myapp-vault-prod \
|
||||
--name app-tls-cert \
|
||||
--file cert.pem \
|
||||
--encoding PEM
|
||||
|
||||
# List certificates
|
||||
az keyvault certificate list --vault-name myapp-vault-prod -o table
|
||||
```
|
||||
|
||||
## Access Policies and RBAC
|
||||
|
||||
### RBAC (Recommended)
|
||||
|
||||
```bash
|
||||
# Grant secret reader access to a managed identity
|
||||
az role assignment create \
|
||||
--role "Key Vault Secrets User" \
|
||||
--assignee-object-id "$(az identity show -g rg-app -n myapp-identity --query principalId -o tsv)" \
|
||||
--scope "/subscriptions/{sub}/resourceGroups/rg-secrets/providers/Microsoft.KeyVault/vaults/myapp-vault-prod"
|
||||
|
||||
# Grant admin access to security team
|
||||
az role assignment create \
|
||||
--role "Key Vault Administrator" \
|
||||
--assignee "security-team@example.com" \
|
||||
--scope "/subscriptions/{sub}/resourceGroups/rg-secrets/providers/Microsoft.KeyVault/vaults/myapp-vault-prod"
|
||||
|
||||
# Available Key Vault RBAC roles:
|
||||
# - Key Vault Administrator (full management)
|
||||
# - Key Vault Secrets Officer (manage secrets)
|
||||
# - Key Vault Secrets User (read secrets)
|
||||
# - Key Vault Certificates Officer (manage certs)
|
||||
# - Key Vault Crypto Officer (manage keys)
|
||||
# - Key Vault Crypto User (use keys for encrypt/decrypt)
|
||||
# - Key Vault Reader (read metadata only)
|
||||
```
|
||||
|
||||
### Access Policies (Legacy)
|
||||
|
||||
```bash
|
||||
# Grant secret access via access policy
|
||||
az keyvault set-policy \
|
||||
--name myapp-vault-prod \
|
||||
--object-id "$(az identity show -g rg-app -n myapp-identity --query principalId -o tsv)" \
|
||||
--secret-permissions get list
|
||||
|
||||
# Grant key access
|
||||
az keyvault set-policy \
|
||||
--name myapp-vault-prod \
|
||||
--object-id "$OBJECT_ID" \
|
||||
--key-permissions get unwrapKey wrapKey
|
||||
|
||||
# Grant certificate access
|
||||
az keyvault set-policy \
|
||||
--name myapp-vault-prod \
|
||||
--object-id "$OBJECT_ID" \
|
||||
--certificate-permissions get list
|
||||
```
|
||||
|
||||
## Application Integration
|
||||
|
||||
### Python SDK
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential
|
||||
from azure.keyvault.secrets import SecretClient
|
||||
from azure.keyvault.keys import KeyClient
|
||||
from azure.keyvault.certificates import CertificateClient
|
||||
|
||||
# Use DefaultAzureCredential (works locally and in Azure)
|
||||
credential = DefaultAzureCredential()
|
||||
client = SecretClient(vault_url="https://mykeyvault.vault.azure.net/", credential=credential)
|
||||
|
||||
# Get secret
|
||||
secret = client.get_secret("db-password")
|
||||
print(secret.value)
|
||||
vault_url = "https://myapp-vault-prod.vault.azure.net/"
|
||||
|
||||
# Secrets
|
||||
secret_client = SecretClient(vault_url=vault_url, credential=credential)
|
||||
db_password = secret_client.get_secret("db-password")
|
||||
print(f"Secret value: {db_password.value}")
|
||||
|
||||
# Get specific version
|
||||
specific = secret_client.get_secret("db-password", version="abc123")
|
||||
|
||||
# List secrets
|
||||
for secret_properties in secret_client.list_properties_of_secrets():
|
||||
print(f"Secret: {secret_properties.name}, Enabled: {secret_properties.enabled}")
|
||||
|
||||
# Keys
|
||||
key_client = KeyClient(vault_url=vault_url, credential=credential)
|
||||
from azure.keyvault.keys.crypto import CryptographyClient, EncryptionAlgorithm
|
||||
|
||||
key = key_client.get_key("data-encryption-key")
|
||||
crypto_client = CryptographyClient(key, credential=credential)
|
||||
|
||||
# Encrypt data
|
||||
plaintext = b"sensitive data"
|
||||
result = crypto_client.encrypt(EncryptionAlgorithm.rsa_oaep_256, plaintext)
|
||||
ciphertext = result.ciphertext
|
||||
|
||||
# Decrypt data
|
||||
decrypted = crypto_client.decrypt(EncryptionAlgorithm.rsa_oaep_256, ciphertext)
|
||||
```
|
||||
|
||||
## Kubernetes Integration
|
||||
### .NET SDK
|
||||
|
||||
```csharp
|
||||
using Azure.Identity;
|
||||
using Azure.Security.KeyVault.Secrets;
|
||||
|
||||
var credential = new DefaultAzureCredential();
|
||||
var client = new SecretClient(new Uri("https://myapp-vault-prod.vault.azure.net/"), credential);
|
||||
|
||||
KeyVaultSecret secret = await client.GetSecretAsync("db-password");
|
||||
string password = secret.Value;
|
||||
```
|
||||
|
||||
## Kubernetes Integration (AKS)
|
||||
|
||||
### Secrets Store CSI Driver
|
||||
|
||||
```yaml
|
||||
# SecretProviderClass for AKS with managed identity
|
||||
apiVersion: secrets-store.csi.x-k8s.io/v1
|
||||
kind: SecretProviderClass
|
||||
metadata:
|
||||
name: azure-keyvault
|
||||
name: azure-keyvault-secrets
|
||||
namespace: production
|
||||
spec:
|
||||
provider: azure
|
||||
parameters:
|
||||
keyvaultName: "mykeyvault"
|
||||
usePodIdentity: "false"
|
||||
useVMManagedIdentity: "true"
|
||||
userAssignedIdentityID: "<managed-identity-client-id>"
|
||||
keyvaultName: "myapp-vault-prod"
|
||||
cloudName: ""
|
||||
objects: |
|
||||
array:
|
||||
- |
|
||||
objectName: db-password
|
||||
objectType: secret
|
||||
tenantId: "tenant-id"
|
||||
objectVersion: ""
|
||||
- |
|
||||
objectName: api-key
|
||||
objectType: secret
|
||||
- |
|
||||
objectName: app-tls-cert
|
||||
objectType: secret
|
||||
tenantId: "<azure-tenant-id>"
|
||||
secretObjects:
|
||||
- secretName: db-secrets
|
||||
type: Opaque
|
||||
data:
|
||||
- objectName: db-password
|
||||
key: password
|
||||
- objectName: api-key
|
||||
key: api-key
|
||||
- secretName: tls-secret
|
||||
type: kubernetes.io/tls
|
||||
data:
|
||||
- objectName: app-tls-cert
|
||||
key: tls.crt
|
||||
---
|
||||
# Pod using the secrets
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: myapp
|
||||
namespace: production
|
||||
spec:
|
||||
serviceAccountName: myapp-sa
|
||||
containers:
|
||||
- name: myapp
|
||||
image: ghcr.io/acme/myapp:v1.0.0
|
||||
env:
|
||||
- name: DB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: db-secrets
|
||||
key: password
|
||||
volumeMounts:
|
||||
- name: secrets-store
|
||||
mountPath: "/mnt/secrets-store"
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: secrets-store
|
||||
csi:
|
||||
driver: secrets-store.csi.k8s.io
|
||||
readOnly: true
|
||||
volumeAttributes:
|
||||
secretProviderClass: "azure-keyvault-secrets"
|
||||
```
|
||||
|
||||
## Terraform Configuration
|
||||
|
||||
```hcl
|
||||
resource "azurerm_key_vault" "main" {
|
||||
name = "myapp-vault-prod"
|
||||
location = azurerm_resource_group.main.location
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
tenant_id = data.azurerm_client_config.current.tenant_id
|
||||
sku_name = "premium"
|
||||
|
||||
enable_rbac_authorization = true
|
||||
purge_protection_enabled = true
|
||||
soft_delete_retention_days = 90
|
||||
public_network_access_enabled = false
|
||||
|
||||
network_acls {
|
||||
bypass = "AzureServices"
|
||||
default_action = "Deny"
|
||||
ip_rules = ["203.0.113.0/24"]
|
||||
virtual_network_subnet_ids = [azurerm_subnet.app.id]
|
||||
}
|
||||
}
|
||||
|
||||
resource "azurerm_key_vault_secret" "db_password" {
|
||||
name = "db-password"
|
||||
value = var.db_password
|
||||
key_vault_id = azurerm_key_vault.main.id
|
||||
content_type = "text/plain"
|
||||
|
||||
expiration_date = "2026-01-01T00:00:00Z"
|
||||
|
||||
tags = {
|
||||
environment = "production"
|
||||
rotation = "enabled"
|
||||
}
|
||||
}
|
||||
|
||||
resource "azurerm_role_assignment" "app_secrets_user" {
|
||||
scope = azurerm_key_vault.main.id
|
||||
role_definition_name = "Key Vault Secrets User"
|
||||
principal_id = azurerm_user_assigned_identity.app.principal_id
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| "Access denied" when reading secrets | Missing RBAC role or access policy | Assign `Key Vault Secrets User` role; or add access policy with `get` permission |
|
||||
| "Vault not found" | Network access restricted | Check firewall rules; enable private endpoint; add IP to allow list |
|
||||
| Soft-deleted secret blocks creation | Name collision with deleted secret | Recover and update, or purge the deleted secret first |
|
||||
| Managed identity cannot access vault | Identity not in correct scope | Verify identity principal ID; check role assignment scope matches vault |
|
||||
| Certificate renewal fails | Auto-renew policy not configured | Set `lifetimeActions` with `AutoRenew` action in certificate policy |
|
||||
| CSI driver fails to mount secrets | Wrong provider configuration | Verify `tenantId`, `userAssignedIdentityID`, and object names match exactly |
|
||||
| High latency on secret retrieval | No client-side caching | Implement caching in application; use CSI driver for K8s (syncs on interval) |
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Use managed identities
|
||||
- Enable soft-delete and purge protection
|
||||
- Implement access policies carefully
|
||||
- Use private endpoints
|
||||
- Monitor with Azure Monitor
|
||||
- Use RBAC authorization over access policies for granular control
|
||||
- Enable soft-delete and purge protection (required for compliance)
|
||||
- Use managed identities for all service access (no credentials to manage)
|
||||
- Enable private endpoints to eliminate public network exposure
|
||||
- Set expiration dates on all secrets and certificates
|
||||
- Enable diagnostic logging and forward to SIEM
|
||||
- Use premium SKU for HSM-backed key operations
|
||||
- Implement key rotation policies for all encryption keys
|
||||
- Regularly audit access with Azure Activity logs
|
||||
- Tag all vault resources for cost and ownership tracking
|
||||
|
||||
## Related Skills
|
||||
|
||||
- [hashicorp-vault](../hashicorp-vault/) - Multi-cloud secrets
|
||||
- [azure-networking](../../../infrastructure/cloud-azure/azure-networking/) - Network security
|
||||
- [aws-secrets-manager](../aws-secrets-manager/) - AWS secret management
|
||||
- [gcp-secret-manager](../gcp-secret-manager/) - GCP secret management
|
||||
|
||||
@@ -14,68 +14,496 @@ Store and manage secrets securely in Google Cloud Platform.
|
||||
## When to Use This Skill
|
||||
|
||||
Use this skill when:
|
||||
- Managing secrets in GCP
|
||||
- Integrating with GKE workloads
|
||||
- Storing API keys and credentials
|
||||
- Implementing secret rotation
|
||||
- Managing secrets in GCP environments
|
||||
- Integrating secrets with GKE workloads via Workload Identity
|
||||
- Storing API keys, database credentials, or TLS certificates
|
||||
- Implementing secret versioning and rotation
|
||||
- Meeting compliance requirements for centralized secret management
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- GCP project
|
||||
- gcloud CLI configured
|
||||
- Secret Manager API enabled
|
||||
- GCP project with billing enabled
|
||||
- `gcloud` CLI installed and authenticated
|
||||
- Secret Manager API enabled (`secretmanager.googleapis.com`)
|
||||
- IAM permissions: `roles/secretmanager.admin` for management, `roles/secretmanager.secretAccessor` for reading
|
||||
- For GKE: Workload Identity configured on the cluster
|
||||
|
||||
## Basic Operations
|
||||
## Enable the API
|
||||
|
||||
```bash
|
||||
# Create secret
|
||||
echo -n "secret123" | gcloud secrets create db-password --data-file=-
|
||||
# Enable Secret Manager API
|
||||
gcloud services enable secretmanager.googleapis.com
|
||||
|
||||
# Access secret
|
||||
# Verify it's enabled
|
||||
gcloud services list --enabled --filter="name:secretmanager"
|
||||
```
|
||||
|
||||
## Secret Creation and Management
|
||||
|
||||
```bash
|
||||
# Create a secret (creates the secret resource, not the value)
|
||||
gcloud secrets create db-password \
|
||||
--replication-policy="automatic" \
|
||||
--labels="env=production,team=platform"
|
||||
|
||||
# Add the secret value (first version)
|
||||
echo -n "S3cur3P@ssw0rd!" | gcloud secrets versions add db-password --data-file=-
|
||||
|
||||
# Create secret with value in one command
|
||||
echo -n '{"username":"dbadmin","password":"S3cur3P@ss!","host":"10.0.1.5","port":5432}' | \
|
||||
gcloud secrets create db-credentials --data-file=- \
|
||||
--replication-policy="automatic" \
|
||||
--labels="env=production,team=platform"
|
||||
|
||||
# Create with specific region replication
|
||||
gcloud secrets create regional-secret \
|
||||
--replication-policy="user-managed" \
|
||||
--locations="us-central1,us-east1"
|
||||
|
||||
# Create with customer-managed encryption key (CMEK)
|
||||
gcloud secrets create sensitive-secret \
|
||||
--replication-policy="user-managed" \
|
||||
--locations="us-central1" \
|
||||
--kms-key-name="projects/my-project/locations/us-central1/keyRings/my-ring/cryptoKeys/my-key"
|
||||
|
||||
# Access the latest version
|
||||
gcloud secrets versions access latest --secret=db-password
|
||||
|
||||
# Add new version
|
||||
echo -n "newsecret" | gcloud secrets versions add db-password --data-file=-
|
||||
# Access a specific version
|
||||
gcloud secrets versions access 3 --secret=db-password
|
||||
|
||||
# List secrets
|
||||
gcloud secrets list
|
||||
# Add a new version (rotation)
|
||||
echo -n "N3wS3cur3P@ss!" | gcloud secrets versions add db-password --data-file=-
|
||||
|
||||
# List all secrets
|
||||
gcloud secrets list --format="table(name, createTime, labels)"
|
||||
|
||||
# List versions of a secret
|
||||
gcloud secrets versions list db-password --format="table(name, state, createTime)"
|
||||
|
||||
# Disable a version (makes it inaccessible but recoverable)
|
||||
gcloud secrets versions disable 1 --secret=db-password
|
||||
|
||||
# Enable a disabled version
|
||||
gcloud secrets versions enable 1 --secret=db-password
|
||||
|
||||
# Destroy a version (permanent)
|
||||
gcloud secrets versions destroy 1 --secret=db-password
|
||||
|
||||
# Delete the entire secret
|
||||
gcloud secrets delete db-password
|
||||
|
||||
# Set expiration on a secret
|
||||
gcloud secrets update db-password \
|
||||
--expire-time="2026-06-01T00:00:00Z"
|
||||
|
||||
# Set TTL-based expiration
|
||||
gcloud secrets update temp-token \
|
||||
--ttl="2592000s" # 30 days
|
||||
|
||||
# Update labels
|
||||
gcloud secrets update db-password \
|
||||
--update-labels="rotation=enabled,last-rotated=2025-01-15"
|
||||
|
||||
# Add version aliases
|
||||
gcloud secrets versions update 5 --secret=db-password --set-aliases="production"
|
||||
```
|
||||
|
||||
## Application Integration
|
||||
## IAM Bindings
|
||||
|
||||
```python
|
||||
from google.cloud import secretmanager
|
||||
```bash
|
||||
# Grant secret accessor role to a service account
|
||||
gcloud secrets add-iam-policy-binding db-password \
|
||||
--member="serviceAccount:myapp-sa@my-project.iam.gserviceaccount.com" \
|
||||
--role="roles/secretmanager.secretAccessor"
|
||||
|
||||
client = secretmanager.SecretManagerServiceClient()
|
||||
name = f"projects/my-project/secrets/db-password/versions/latest"
|
||||
response = client.access_secret_version(request={"name": name})
|
||||
secret = response.payload.data.decode("UTF-8")
|
||||
# Grant access to a specific secret version
|
||||
gcloud secrets add-iam-policy-binding db-password \
|
||||
--member="serviceAccount:myapp-sa@my-project.iam.gserviceaccount.com" \
|
||||
--role="roles/secretmanager.secretVersionAccessor" \
|
||||
--condition='expression=resource.name.endsWith("versions/latest"),title=latest-only'
|
||||
|
||||
# Grant admin to security team
|
||||
gcloud secrets add-iam-policy-binding db-password \
|
||||
--member="group:security-team@example.com" \
|
||||
--role="roles/secretmanager.admin"
|
||||
|
||||
# View IAM policy for a secret
|
||||
gcloud secrets get-iam-policy db-password
|
||||
|
||||
# Remove access
|
||||
gcloud secrets remove-iam-policy-binding db-password \
|
||||
--member="serviceAccount:old-sa@my-project.iam.gserviceaccount.com" \
|
||||
--role="roles/secretmanager.secretAccessor"
|
||||
|
||||
# Project-level IAM for all secrets
|
||||
gcloud projects add-iam-policy-binding my-project \
|
||||
--member="serviceAccount:myapp-sa@my-project.iam.gserviceaccount.com" \
|
||||
--role="roles/secretmanager.secretAccessor" \
|
||||
--condition='expression=resource.name.startsWith("projects/my-project/secrets/myapp-"),title=myapp-secrets-only'
|
||||
```
|
||||
|
||||
## GKE Integration
|
||||
## Workload Identity for GKE
|
||||
|
||||
```bash
|
||||
# Enable Workload Identity on cluster (if not already)
|
||||
gcloud container clusters update my-cluster \
|
||||
--zone us-central1-a \
|
||||
--workload-pool=my-project.svc.id.goog
|
||||
|
||||
# Create GCP service account for the workload
|
||||
gcloud iam service-accounts create myapp-gke-sa \
|
||||
--display-name="MyApp GKE Service Account"
|
||||
|
||||
# Grant secret accessor role
|
||||
gcloud secrets add-iam-policy-binding db-password \
|
||||
--member="serviceAccount:myapp-gke-sa@my-project.iam.gserviceaccount.com" \
|
||||
--role="roles/secretmanager.secretAccessor"
|
||||
|
||||
# Bind Kubernetes SA to GCP SA
|
||||
gcloud iam service-accounts add-iam-policy-binding \
|
||||
myapp-gke-sa@my-project.iam.gserviceaccount.com \
|
||||
--role="roles/iam.workloadIdentityUser" \
|
||||
--member="serviceAccount:my-project.svc.id.goog[production/myapp-sa]"
|
||||
```
|
||||
|
||||
### Kubernetes Manifests
|
||||
|
||||
```yaml
|
||||
# Kubernetes service account annotated with GCP SA
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: myapp-sa
|
||||
namespace: production
|
||||
annotations:
|
||||
iam.gke.io/gcp-service-account: "myapp-gke-sa@my-project.iam.gserviceaccount.com"
|
||||
---
|
||||
# Secrets Store CSI Driver for GCP
|
||||
apiVersion: secrets-store.csi.x-k8s.io/v1
|
||||
kind: SecretProviderClass
|
||||
metadata:
|
||||
name: gcp-secrets
|
||||
namespace: production
|
||||
spec:
|
||||
provider: gcp
|
||||
parameters:
|
||||
secrets: |
|
||||
- resourceName: "projects/my-project/secrets/db-password/versions/latest"
|
||||
path: "db-password"
|
||||
- resourceName: "projects/my-project/secrets/db-credentials/versions/latest"
|
||||
path: "db-credentials"
|
||||
- resourceName: "projects/my-project/secrets/api-key/versions/latest"
|
||||
path: "api-key"
|
||||
secretObjects:
|
||||
- secretName: myapp-secrets
|
||||
type: Opaque
|
||||
data:
|
||||
- objectName: db-password
|
||||
key: DB_PASSWORD
|
||||
- objectName: api-key
|
||||
key: API_KEY
|
||||
---
|
||||
# Deployment using the secrets
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: myapp
|
||||
namespace: production
|
||||
spec:
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: myapp
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: myapp
|
||||
spec:
|
||||
serviceAccountName: myapp-sa
|
||||
containers:
|
||||
- name: myapp
|
||||
image: gcr.io/my-project/myapp:v1.0.0
|
||||
env:
|
||||
- name: DB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: myapp-secrets
|
||||
key: DB_PASSWORD
|
||||
volumeMounts:
|
||||
- name: secrets
|
||||
mountPath: "/var/secrets"
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: secrets
|
||||
csi:
|
||||
driver: secrets-store.csi.k8s.io
|
||||
readOnly: true
|
||||
volumeAttributes:
|
||||
secretProviderClass: "gcp-secrets"
|
||||
```
|
||||
|
||||
## Application SDK Examples
|
||||
|
||||
### Python
|
||||
|
||||
```python
|
||||
from google.cloud import secretmanager
|
||||
from google.api_core import exceptions
|
||||
import json
|
||||
|
||||
def get_secret(project_id: str, secret_id: str, version: str = "latest") -> str:
|
||||
"""Access a secret version from GCP Secret Manager."""
|
||||
client = secretmanager.SecretManagerServiceClient()
|
||||
name = f"projects/{project_id}/secrets/{secret_id}/versions/{version}"
|
||||
|
||||
try:
|
||||
response = client.access_secret_version(request={"name": name})
|
||||
return response.payload.data.decode("UTF-8")
|
||||
except exceptions.NotFound:
|
||||
raise ValueError(f"Secret {secret_id} version {version} not found")
|
||||
except exceptions.PermissionDenied:
|
||||
raise PermissionError(f"No access to secret {secret_id}")
|
||||
|
||||
def get_json_secret(project_id: str, secret_id: str) -> dict:
|
||||
"""Access and parse a JSON secret."""
|
||||
raw = get_secret(project_id, secret_id)
|
||||
return json.loads(raw)
|
||||
|
||||
def create_secret(project_id: str, secret_id: str, value: str, labels: dict = None) -> str:
|
||||
"""Create a new secret with an initial version."""
|
||||
client = secretmanager.SecretManagerServiceClient()
|
||||
parent = f"projects/{project_id}"
|
||||
|
||||
secret_config = {
|
||||
"replication": {"automatic": {}},
|
||||
}
|
||||
if labels:
|
||||
secret_config["labels"] = labels
|
||||
|
||||
secret = client.create_secret(
|
||||
request={"parent": parent, "secret_id": secret_id, "secret": secret_config}
|
||||
)
|
||||
|
||||
client.add_secret_version(
|
||||
request={"parent": secret.name, "payload": {"data": value.encode("UTF-8")}}
|
||||
)
|
||||
return secret.name
|
||||
|
||||
def rotate_secret(project_id: str, secret_id: str, new_value: str) -> str:
|
||||
"""Add a new version to rotate the secret."""
|
||||
client = secretmanager.SecretManagerServiceClient()
|
||||
parent = f"projects/{project_id}/secrets/{secret_id}"
|
||||
|
||||
version = client.add_secret_version(
|
||||
request={"parent": parent, "payload": {"data": new_value.encode("UTF-8")}}
|
||||
)
|
||||
return version.name
|
||||
|
||||
def list_secrets(project_id: str, filter_str: str = "") -> list:
|
||||
"""List all secrets in a project."""
|
||||
client = secretmanager.SecretManagerServiceClient()
|
||||
parent = f"projects/{project_id}"
|
||||
|
||||
secrets = []
|
||||
for secret in client.list_secrets(request={"parent": parent, "filter": filter_str}):
|
||||
secrets.append({
|
||||
"name": secret.name.split("/")[-1],
|
||||
"created": secret.create_time.isoformat(),
|
||||
"labels": dict(secret.labels),
|
||||
})
|
||||
return secrets
|
||||
|
||||
# Usage
|
||||
creds = get_json_secret("my-project", "db-credentials")
|
||||
connection_string = (
|
||||
f"postgresql://{creds['username']}:{creds['password']}"
|
||||
f"@{creds['host']}:{creds['port']}/mydb"
|
||||
)
|
||||
```
|
||||
|
||||
### Go
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
secretmanager "cloud.google.com/go/secretmanager/apiv1"
|
||||
secretmanagerpb "cloud.google.com/go/secretmanager/apiv1/secretmanagerpb"
|
||||
)
|
||||
|
||||
func getSecret(projectID, secretID, version string) (string, error) {
|
||||
ctx := context.Background()
|
||||
client, err := secretmanager.NewClient(ctx)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to create client: %w", err)
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
name := fmt.Sprintf("projects/%s/secrets/%s/versions/%s", projectID, secretID, version)
|
||||
result, err := client.AccessSecretVersion(ctx, &secretmanagerpb.AccessSecretVersionRequest{
|
||||
Name: name,
|
||||
})
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to access secret: %w", err)
|
||||
}
|
||||
|
||||
return string(result.Payload.Data), nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
secret, err := getSecret("my-project", "db-password", "latest")
|
||||
if err != nil {
|
||||
log.Fatalf("Error: %v", err)
|
||||
}
|
||||
fmt.Printf("Secret: %s\n", secret)
|
||||
}
|
||||
```
|
||||
|
||||
### Node.js
|
||||
|
||||
```javascript
|
||||
const { SecretManagerServiceClient } = require('@google-cloud/secret-manager');
|
||||
|
||||
const client = new SecretManagerServiceClient();
|
||||
|
||||
async function getSecret(projectId, secretId, version = 'latest') {
|
||||
const name = `projects/${projectId}/secrets/${secretId}/versions/${version}`;
|
||||
const [response] = await client.accessSecretVersion({ name });
|
||||
return response.payload.data.toString('utf8');
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const password = await getSecret('my-project', 'db-password');
|
||||
console.log(`Secret retrieved, length: ${password.length}`);
|
||||
}
|
||||
|
||||
main().catch(console.error);
|
||||
```
|
||||
|
||||
## Secret Rotation with Cloud Functions
|
||||
|
||||
```python
|
||||
"""cloud_function_rotation.py - Triggered by Pub/Sub on secret rotation events."""
|
||||
|
||||
import functions_framework
|
||||
from google.cloud import secretmanager
|
||||
import secrets
|
||||
import string
|
||||
|
||||
@functions_framework.cloud_event
|
||||
def rotate_secret(cloud_event):
|
||||
"""Handle secret rotation events from Pub/Sub."""
|
||||
data = cloud_event.data
|
||||
secret_name = data.get("name", "")
|
||||
|
||||
if "db-password" not in secret_name:
|
||||
print(f"Skipping non-DB secret: {secret_name}")
|
||||
return
|
||||
|
||||
client = secretmanager.SecretManagerServiceClient()
|
||||
|
||||
alphabet = string.ascii_letters + string.digits + "!@#$%^&*"
|
||||
new_password = ''.join(secrets.choice(alphabet) for _ in range(32))
|
||||
|
||||
parent = "/".join(secret_name.split("/")[:4])
|
||||
client.add_secret_version(
|
||||
request={
|
||||
"parent": parent,
|
||||
"payload": {"data": new_password.encode("UTF-8")},
|
||||
}
|
||||
)
|
||||
print(f"Rotated secret: {parent}")
|
||||
```
|
||||
|
||||
### Rotation Schedule with Cloud Scheduler
|
||||
|
||||
```bash
|
||||
# Create a Pub/Sub topic for rotation events
|
||||
gcloud pubsub topics create secret-rotation
|
||||
|
||||
# Configure secret to publish rotation events
|
||||
gcloud secrets update db-password \
|
||||
--add-topics="projects/my-project/topics/secret-rotation" \
|
||||
--event-types="SECRET_ROTATE"
|
||||
|
||||
# Set up rotation schedule
|
||||
gcloud secrets update db-password \
|
||||
--next-rotation-time="2025-04-01T00:00:00Z" \
|
||||
--rotation-period="2592000s" # 30 days
|
||||
```
|
||||
|
||||
## Terraform Configuration
|
||||
|
||||
```hcl
|
||||
resource "google_secret_manager_secret" "db_password" {
|
||||
project = var.project_id
|
||||
secret_id = "db-password"
|
||||
|
||||
replication {
|
||||
auto {}
|
||||
}
|
||||
|
||||
labels = {
|
||||
env = "production"
|
||||
team = "platform"
|
||||
}
|
||||
|
||||
rotation {
|
||||
next_rotation_time = "2025-04-01T00:00:00Z"
|
||||
rotation_period = "2592000s"
|
||||
}
|
||||
|
||||
topics {
|
||||
name = google_pubsub_topic.secret_rotation.id
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_secret_manager_secret_version" "db_password" {
|
||||
secret = google_secret_manager_secret.db_password.id
|
||||
secret_data = var.db_password
|
||||
}
|
||||
|
||||
resource "google_secret_manager_secret_iam_member" "app_accessor" {
|
||||
project = var.project_id
|
||||
secret_id = google_secret_manager_secret.db_password.secret_id
|
||||
role = "roles/secretmanager.secretAccessor"
|
||||
member = "serviceAccount:${google_service_account.app.email}"
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| "Secret Manager API not enabled" | API not activated in project | Run `gcloud services enable secretmanager.googleapis.com` |
|
||||
| "Permission denied" on access | Missing `secretAccessor` role | Grant `roles/secretmanager.secretAccessor` on the specific secret |
|
||||
| Workload Identity not working | K8s SA not bound to GCP SA | Verify annotation on K8s SA; check IAM binding with `workloadIdentityUser` |
|
||||
| "Secret version is in DISABLED state" | Version was disabled | Enable with `gcloud secrets versions enable VERSION --secret=SECRET` |
|
||||
| High latency on secret access | No client-side caching | Cache secrets in memory with TTL; use CSI driver for GKE |
|
||||
| CMEK decrypt fails | KMS key permissions missing | Grant `roles/cloudkms.cryptoKeyEncrypterDecrypter` to Secret Manager SA |
|
||||
| Rotation function not triggered | Pub/Sub topic not configured | Verify topic is attached to secret; check Cloud Function subscription |
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Use Workload Identity for GKE
|
||||
- Implement IAM least-privilege
|
||||
- Enable audit logging
|
||||
- Use secret versions for rollback
|
||||
- Integrate with Cloud KMS for encryption
|
||||
- Use Workload Identity for GKE instead of exported service account keys
|
||||
- Implement IAM least-privilege at the individual secret level, not project level
|
||||
- Enable audit logging for all secret access (Cloud Audit Logs)
|
||||
- Use secret versions for safe rollback during rotation issues
|
||||
- Set expiration dates or TTLs on temporary secrets
|
||||
- Integrate with Cloud KMS for customer-managed encryption keys
|
||||
- Use labels consistently for organization and automation
|
||||
- Monitor secret access patterns with Cloud Monitoring
|
||||
- Implement rotation schedules for all long-lived credentials
|
||||
- Use conditional IAM bindings to restrict access by resource name pattern
|
||||
|
||||
## Related Skills
|
||||
|
||||
- [hashicorp-vault](../hashicorp-vault/) - Multi-cloud secrets
|
||||
- [gcp-gke](../../../infrastructure/cloud-gcp/gcp-gke/) - GKE integration
|
||||
- [aws-secrets-manager](../aws-secrets-manager/) - AWS secret management
|
||||
- [azure-keyvault](../azure-keyvault/) - Azure secret management
|
||||
|
||||
Reference in New Issue
Block a user