External/DevOps-Security-Agent-Skills
1
0
Fork 0
mirror of https://github.com/BagelHole/DevOps-Security-Agent-Skills.git synced 2026-08-22 12:49:53 +02:00
Code Issues Packages Projects Releases Wiki Activity

V2

Browse Source
This commit is contained in:
Toby
2026-03-24 18:02:50 -04:00
parent 2d209b9258
commit ba9e489584
111 changed files with 48382 additions and 3099 deletions
Download Patch File Download Diff File Expand all files Collapse all files
infrastructure/cloud-azure/arm-templates/SKILL.md
+429 -14
View File
@@ -9,50 +9,465 @@ metadata:
# ARM Templates & Bicep
Deploy Azure infrastructure with ARM templates and Bicep.
Deploy Azure infrastructure with ARM templates and Bicep. Bicep is the recommended domain-specific language that compiles to ARM JSON, offering cleaner syntax, modules, and first-class tooling support.
## Bicep Example
## When to Use
- You need Azure-native Infrastructure as Code without third-party tooling.
- Your organization standardizes on Azure and wants tight portal integration.
- You need What-If analysis before deploying changes.
- You are migrating existing ARM JSON templates to Bicep for maintainability.
- You need deployment scopes at resource group, subscription, management group, or tenant level.
## Prerequisites
```bash
# Install Azure CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
# Install Bicep CLI (bundled with Azure CLI 2.20+)
az bicep install
az bicep upgrade
# Verify installation
az bicep version
# Login and set subscription
az login
az account set --subscription "my-subscription-id"
```
## Bicep Fundamentals
### Resource Group Deployment with Virtual Network
```bicep
// main.bicep
@description('Azure region for all resources')
param location string = resourceGroup().location
@description('Environment name used for resource naming')
@allowed(['dev', 'staging', 'prod'])
param environment string = 'dev'
@description('Base name for all resources')
param baseName string
var vnetName = '${baseName}-${environment}-vnet'
var nsgName = '${baseName}-${environment}-nsg'
resource nsg 'Microsoft.Network/networkSecurityGroups@2023-05-01' = {
name: nsgName
location: location
properties: {
securityRules: [
{
name: 'AllowHTTPS'
properties: {
priority: 100
direction: 'Inbound'
access: 'Allow'
protocol: 'Tcp'
sourcePortRange: '*'
destinationPortRange: '443'
sourceAddressPrefix: '*'
destinationAddressPrefix: '*'
}
}
{
name: 'DenyAllInbound'
properties: {
priority: 4096
direction: 'Inbound'
access: 'Deny'
protocol: '*'
sourcePortRange: '*'
destinationPortRange: '*'
sourceAddressPrefix: '*'
destinationAddressPrefix: '*'
}
}
]
}
}
resource vnet 'Microsoft.Network/virtualNetworks@2023-05-01' = {
name: vnetName
location: location
properties: {
addressSpace: {
addressPrefixes: [
'10.0.0.0/16'
]
}
subnets: [
{
name: 'web-subnet'
properties: {
addressPrefix: '10.0.1.0/24'
networkSecurityGroup: {
id: nsg.id
}
}
}
{
name: 'app-subnet'
properties: {
addressPrefix: '10.0.2.0/24'
}
}
{
name: 'data-subnet'
properties: {
addressPrefix: '10.0.3.0/24'
privateEndpointNetworkPolicies: 'Enabled'
}
}
]
}
}
output vnetId string = vnet.id
output webSubnetId string = vnet.properties.subnets[0].id
output appSubnetId string = vnet.properties.subnets[1].id
```
### VM Deployment with Managed Identity
```bicep
// vm.bicep
param location string = resourceGroup().location
param vmName string
param subnetId string
param adminUsername string = 'azureuser'
resource vm 'Microsoft.Compute/virtualMachines@2023-03-01' = {
@secure()
param adminPublicKey string
resource nic 'Microsoft.Network/networkInterfaces@2023-05-01' = {
name: '${vmName}-nic'
location: location
properties: {
ipConfigurations: [
{
name: 'ipconfig1'
properties: {
privateIPAllocationMethod: 'Dynamic'
subnet: {
id: subnetId
}
}
}
]
}
}
resource vm 'Microsoft.Compute/virtualMachines@2023-07-01' = {
name: vmName
location: location
identity: {
type: 'SystemAssigned'
}
properties: {
hardwareProfile: {
vmSize: 'Standard_B2s'
}
osProfile: {
computerName: vmName
adminUsername: 'azureuser'
adminUsername: adminUsername
linuxConfiguration: {
disablePasswordAuthentication: true
ssh: {
publicKeys: [
{
path: '/home/${adminUsername}/.ssh/authorized_keys'
keyData: adminPublicKey
}
]
}
}
}
storageProfile: {
imageReference: {
publisher: 'Canonical'
offer: '0001-com-ubuntu-server-jammy'
sku: '22_04-lts-gen2'
version: 'latest'
}
osDisk: {
createOption: 'FromImage'
managedDisk: {
storageAccountType: 'Premium_LRS'
}
}
}
networkProfile: {
networkInterfaces: [
{
id: nic.id
}
]
}
diagnosticsProfile: {
bootDiagnostics: {
enabled: true
}
}
}
}
output vmPrincipalId string = vm.identity.principalId
output vmId string = vm.id
```
## Deployment
## Bicep Modules
### Module Definition
```bicep
// modules/storage.bicep
@description('Storage account name (3-24 chars, lowercase alphanumeric)')
param storageAccountName string
param location string = resourceGroup().location
param sku string = 'Standard_LRS'
@allowed(['Hot', 'Cool', 'Archive'])
param accessTier string = 'Hot'
resource storageAccount 'Microsoft.Storage/storageAccounts@2023-01-01' = {
name: storageAccountName
location: location
sku: {
name: sku
}
kind: 'StorageV2'
properties: {
accessTier: accessTier
supportsHttpsTrafficOnly: true
minimumTlsVersion: 'TLS1_2'
allowBlobPublicAccess: false
networkAcls: {
defaultAction: 'Deny'
bypass: 'AzureServices'
}
}
}
output storageAccountId string = storageAccount.id
output primaryBlobEndpoint string = storageAccount.properties.primaryEndpoints.blob
```
### Consuming Modules
```bicep
// main.bicep
param location string = resourceGroup().location
param environment string = 'prod'
module storage 'modules/storage.bicep' = {
name: 'storage-deployment'
params: {
storageAccountName: 'myapp${environment}sa'
location: location
sku: environment == 'prod' ? 'Standard_GRS' : 'Standard_LRS'
}
}
module vnet 'modules/network.bicep' = {
name: 'vnet-deployment'
params: {
location: location
environment: environment
}
}
// Reference module outputs
output storageBlobEndpoint string = storage.outputs.primaryBlobEndpoint
```
## ARM JSON Template Structure
```json
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageAccountName": {
"type": "string",
"metadata": {
"description": "Name of the storage account"
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]"
}
},
"variables": {
"storageSku": "Standard_LRS"
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2023-01-01",
"name": "[parameters('storageAccountName')]",
"location": "[parameters('location')]",
"sku": {
"name": "[variables('storageSku')]"
},
"kind": "StorageV2",
"properties": {
"supportsHttpsTrafficOnly": true,
"minimumTlsVersion": "TLS1_2"
}
}
],
"outputs": {
"storageId": {
"type": "string",
"value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
}
}
}
```
## Deployment Commands
```bash
# Deploy Bicep
# Validate a Bicep template before deployment
az deployment group validate \
--resource-group mygroup \
--template-file main.bicep \
--parameters environment='prod' baseName='myapp'
# Preview changes with What-If
az deployment group what-if \
--resource-group mygroup \
--template-file main.bicep \
--parameters environment='prod' baseName='myapp'
# Deploy Bicep to resource group
az deployment group create \
--resource-group mygroup \
--template-file main.bicep \
--parameters vmName=myvm
--parameters environment='prod' baseName='myapp' \
--name "deploy-$(date +%Y%m%d-%H%M%S)"
# Deploy ARM
# Deploy ARM JSON with parameter file
az deployment group create \
--resource-group mygroup \
--template-file template.json \
--parameters @parameters.json
--parameters @parameters.prod.json
# Subscription-level deployment (e.g., resource groups, policies)
az deployment sub create \
--location eastus \
--template-file subscription-level.bicep \
--parameters @params.json
# Management group deployment
az deployment mg create \
--management-group-id my-mg \
--location eastus \
--template-file mg-policy.bicep
# Export resource group to ARM JSON
az group export --name mygroup --output json > exported-template.json
# Decompile ARM JSON to Bicep
az bicep decompile --file exported-template.json
# Build Bicep to ARM JSON (for inspection)
az bicep build --file main.bicep --outfile main.json
# List deployments and their status
az deployment group list \
--resource-group mygroup \
--output table
# Delete a failed deployment
az deployment group delete \
--resource-group mygroup \
--name my-failed-deployment
```
## Best Practices
## Parameter Files
- Use Bicep over JSON ARM
- Implement modules for reusability
- Use parameter files per environment
- Validate before deployment
```json
// parameters.prod.json
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"environment": { "value": "prod" },
"baseName": { "value": "myapp" },
"adminPublicKey": {
"reference": {
"keyVault": {
"id": "/subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.KeyVault/vaults/{vault}"
},
"secretName": "ssh-public-key"
}
}
}
}
```
## Linked and Nested Templates
```bicep
// Deploy to a different resource group
module networkInSharedRg 'modules/network.bicep' = {
name: 'shared-network'
scope: resourceGroup('shared-networking-rg')
params: {
location: location
}
}
// Conditional deployment
param deployMonitoring bool = true
module monitoring 'modules/monitoring.bicep' = if (deployMonitoring) {
name: 'monitoring-deployment'
params: {
location: location
}
}
// Loop deployment
param storageAccounts array = [
{ name: 'logs', sku: 'Standard_LRS' }
{ name: 'data', sku: 'Standard_GRS' }
]
module storageLoop 'modules/storage.bicep' = [for account in storageAccounts: {
name: 'storage-${account.name}'
params: {
storageAccountName: '${baseName}${account.name}sa'
sku: account.sku
location: location
}
}]
```
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| `InvalidTemplate` error | Syntax error in ARM JSON or Bicep | Run `az bicep build` to check for compile errors |
| `ResourceNotFound` during deployment | Resource dependency not declared | Add `dependsOn` or use implicit references in Bicep |
| `DeploymentFailed` with quota error | Subscription quota exceeded | Request quota increase or use a different region |
| `AuthorizationFailed` | Insufficient RBAC permissions | Assign Contributor role on the target resource group |
| Parameter file secrets in source control | Secrets stored as plain text | Use Key Vault references in parameter files |
| Deployment takes very long | Large number of resources deployed serially | Use `dependsOn` carefully to allow parallel deployment |
| `What-If` shows unexpected deletions | Complete mode instead of Incremental | Use `--mode Incremental` (the default) to avoid deleting unmanaged resources |
| Bicep module not found | Incorrect relative path | Verify path is relative to the consuming file |
## Related Skills
- `terraform-azure` -- Multi-cloud IaC alternative with broader provider support.
- `azure-networking` -- VNet, NSG, and firewall configurations referenced in templates.
- `azure-vms` -- Virtual machine sizing and configuration details.
- `azure-aks` -- Kubernetes cluster definitions for Bicep/ARM.
infrastructure/cloud-azure/azure-aks/SKILL.md
+374 -29
View File
@@ -9,54 +9,399 @@ metadata:
# Azure Kubernetes Service
Deploy managed Kubernetes clusters on Azure.
Deploy and manage production-grade Kubernetes clusters on Azure with AKS. Covers cluster creation, node pool management, networking, ingress controllers, monitoring, security, and Terraform-based provisioning.
## Create Cluster
## When to Use
- You need managed Kubernetes without maintaining control plane infrastructure.
- Your workloads require container orchestration with auto-scaling.
- You need tight integration with Azure AD, Key Vault, and Container Registry.
- You are running microservices that require service mesh, ingress, or network policies.
- You need GPU or spot node pools for specialized or cost-optimized workloads.
## Prerequisites
```bash
# Install Azure CLI and kubectl
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
az aks install-cli
# Login and set subscription
az login
az account set --subscription "my-subscription-id"
# Register required providers
az provider register --namespace Microsoft.ContainerService
az provider register --namespace Microsoft.OperationsManagement
# Verify kubectl
kubectl version --client
```
## Cluster Creation
### Basic Production Cluster
```bash
# Create resource group
az group create --name myapp-rg --location eastus
# Create AKS cluster with best-practice defaults
az aks create \
--resource-group myapp-rg \
--name myapp-aks \
--node-count 3 \
--node-vm-size Standard_D4s_v5 \
--enable-managed-identity \
--enable-cluster-autoscaler \
--min-count 2 \
--max-count 10 \
--network-plugin azure \
--network-policy calico \
--service-cidr 10.1.0.0/16 \
--dns-service-ip 10.1.0.10 \
--vnet-subnet-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Network/virtualNetworks/{vnet}/subnets/{subnet}" \
--enable-aad \
--aad-admin-group-object-ids "{aad-group-id}" \
--enable-azure-rbac \
--zones 1 2 3 \
--generate-ssh-keys \
--tags environment=prod team=platform
# Get cluster credentials
az aks get-credentials --resource-group myapp-rg --name myapp-aks
# Verify cluster access
kubectl get nodes -o wide
kubectl cluster-info
```
### Private Cluster
```bash
az aks create \
--resource-group mygroup \
--name myakscluster \
--resource-group myapp-rg \
--name myapp-private-aks \
--node-count 3 \
--node-vm-size Standard_B2s \
--node-vm-size Standard_D4s_v5 \
--enable-managed-identity \
--enable-private-cluster \
--private-dns-zone system \
--network-plugin azure \
--vnet-subnet-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Network/virtualNetworks/{vnet}/subnets/{subnet}" \
--generate-ssh-keys
# Get credentials
az aks get-credentials --resource-group mygroup --name myakscluster
```
## Node Pools
## Node Pool Management
```bash
# Add a user node pool for application workloads
az aks nodepool add \
--resource-group mygroup \
--cluster-name myakscluster \
--resource-group myapp-rg \
--cluster-name myapp-aks \
--name apppool \
--node-count 3 \
--node-vm-size Standard_D8s_v5 \
--mode User \
--enable-cluster-autoscaler \
--min-count 2 \
--max-count 15 \
--zones 1 2 3 \
--labels workload=app tier=frontend \
--node-taints dedicated=app:NoSchedule \
--max-pods 50
# Add GPU node pool for ML workloads
az aks nodepool add \
--resource-group myapp-rg \
--cluster-name myapp-aks \
--name gpupool \
--node-count 1 \
--node-vm-size Standard_NC6
--node-vm-size Standard_NC6s_v3 \
--mode User \
--enable-cluster-autoscaler \
--min-count 0 \
--max-count 4 \
--node-taints sku=gpu:NoSchedule \
--labels workload=ml
# Add spot instance pool for batch workloads
az aks nodepool add \
--resource-group myapp-rg \
--cluster-name myapp-aks \
--name spotpool \
--node-count 2 \
--node-vm-size Standard_D4s_v5 \
--priority Spot \
--eviction-policy Delete \
--spot-max-price -1 \
--enable-cluster-autoscaler \
--min-count 0 \
--max-count 20 \
--labels workload=batch
# Scale a node pool manually
az aks nodepool scale \
--resource-group myapp-rg \
--cluster-name myapp-aks \
--name apppool \
--node-count 5
# Upgrade a node pool
az aks nodepool upgrade \
--resource-group myapp-rg \
--cluster-name myapp-aks \
--name apppool \
--kubernetes-version 1.28.3
# List node pools
az aks nodepool list \
--resource-group myapp-rg \
--cluster-name myapp-aks \
--output table
```
## Enable Add-ons
## Ingress Controller Setup
```bash
# Enable monitoring
az aks enable-addons \
--resource-group mygroup \
--name myakscluster \
--addons monitoring
# Install NGINX ingress controller via Helm
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
# Enable Azure Policy
az aks enable-addons \
--resource-group mygroup \
--name myakscluster \
--addons azure-policy
helm install ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--create-namespace \
--set controller.replicaCount=2 \
--set controller.nodeSelector."kubernetes\.io/os"=linux \
--set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz \
--set controller.service.externalTrafficPolicy=Local
# Verify the ingress controller and get external IP
kubectl get svc -n ingress-nginx
```
## Best Practices
### Ingress Resource Example
- Use managed identity
- Enable Azure CNI for networking
- Implement pod identity
- Use node pools for workload isolation
- Enable cluster autoscaler
```yaml
# ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-ingress
namespace: myapp
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "50m"
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: nginx
tls:
- hosts:
- myapp.example.com
secretName: myapp-tls
rules:
- host: myapp.example.com
http:
paths:
- path: /api
pathType: Prefix
backend:
service:
name: api-service
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: frontend-service
port:
number: 80
```
## Monitoring and Logging
```bash
# Enable Container Insights
az aks enable-addons \
--resource-group myapp-rg \
--name myapp-aks \
--addons monitoring \
--workspace-resource-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.OperationalInsights/workspaces/{workspace}"
# Enable Azure Policy add-on
az aks enable-addons \
--resource-group myapp-rg \
--name myapp-aks \
--addons azure-policy
# Enable Key Vault secrets provider
az aks enable-addons \
--resource-group myapp-rg \
--name myapp-aks \
--addons azure-keyvault-secrets-provider
# View cluster diagnostics
az aks show \
--resource-group myapp-rg \
--name myapp-aks \
--query "addonProfiles" \
--output table
# Install Prometheus + Grafana via Helm
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install kube-prometheus prometheus-community/kube-prometheus-stack \
--namespace monitoring \
--create-namespace \
--set grafana.adminPassword='SecureGrafanaP@ss'
```
## ACR Integration
```bash
# Create Azure Container Registry
az acr create \
--resource-group myapp-rg \
--name myappacr \
--sku Standard
# Attach ACR to AKS (grants AcrPull role)
az aks update \
--resource-group myapp-rg \
--name myapp-aks \
--attach-acr myappacr
# Build and push image
az acr build \
--registry myappacr \
--image myapp:v1.0 \
--file Dockerfile .
# Verify pull access
kubectl run test --image=myappacr.azurecr.io/myapp:v1.0 --rm -it --restart=Never -- echo "ACR pull works"
```
## Terraform Configuration
```hcl
resource "azurerm_kubernetes_cluster" "aks" {
name = "myapp-aks"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
dns_prefix = "myapp"
kubernetes_version = "1.28"
default_node_pool {
name = "system"
vm_size = "Standard_D4s_v5"
enable_auto_scaling = true
min_count = 2
max_count = 5
zones = [1, 2, 3]
vnet_subnet_id = azurerm_subnet.aks.id
node_labels = {
role = "system"
}
}
identity {
type = "SystemAssigned"
}
network_profile {
network_plugin = "azure"
network_policy = "calico"
service_cidr = "10.1.0.0/16"
dns_service_ip = "10.1.0.10"
load_balancer_sku = "standard"
}
azure_active_directory_role_based_access_control {
managed = true
azure_rbac_enabled = true
admin_group_object_ids = [var.aks_admin_group_id]
}
oms_agent {
log_analytics_workspace_id = azurerm_log_analytics_workspace.main.id
}
key_vault_secrets_provider {
secret_rotation_enabled = true
}
tags = var.tags
}
resource "azurerm_kubernetes_cluster_node_pool" "app" {
name = "app"
kubernetes_cluster_id = azurerm_kubernetes_cluster.aks.id
vm_size = "Standard_D8s_v5"
enable_auto_scaling = true
min_count = 2
max_count = 15
zones = [1, 2, 3]
vnet_subnet_id = azurerm_subnet.aks.id
node_labels = {
workload = "app"
}
node_taints = [
"dedicated=app:NoSchedule"
]
tags = var.tags
}
```
## Cluster Upgrades
```bash
# Check available Kubernetes versions
az aks get-upgrades \
--resource-group myapp-rg \
--name myapp-aks \
--output table
# Upgrade control plane first
az aks upgrade \
--resource-group myapp-rg \
--name myapp-aks \
--kubernetes-version 1.28.3 \
--control-plane-only
# Then upgrade each node pool
az aks nodepool upgrade \
--resource-group myapp-rg \
--cluster-name myapp-aks \
--name apppool \
--kubernetes-version 1.28.3
# Check upgrade status
az aks show \
--resource-group myapp-rg \
--name myapp-aks \
--query "provisioningState"
```
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| Nodes in `NotReady` state | VM resource exhaustion or network issues | Run `kubectl describe node <name>` and check events; scale up if needed |
| Pods stuck in `Pending` | No available nodes or resource requests too high | Check autoscaler status with `az aks show`; adjust resource requests |
| `ImagePullBackOff` error | ACR not attached or image tag wrong | Verify with `az aks check-acr --name myapp-aks --acr myappacr.azurecr.io` |
| Ingress returns 404 | Service or path mismatch in Ingress spec | Verify `kubectl get ingress` and service endpoints |
| Private cluster unreachable | No VPN or private endpoint configured | Use `az aks command invoke` or configure private DNS resolution |
| Cluster autoscaler not scaling | Pod resource requests not set | Define CPU/memory requests on all pods so the scheduler can calculate demand |
| Azure Policy violations blocking pods | Restrictive policies applied | Check `kubectl get constrainttemplate` and adjust policy assignments |
| Persistent volume not binding | StorageClass mismatch or zone issue | Verify `kubectl get pvc` and ensure StorageClass matches node pool zones |
## Related Skills
- `terraform-azure` -- Provision AKS clusters with Terraform for repeatable infrastructure.
- `azure-networking` -- VNet and subnet configuration required by Azure CNI.
- `arm-templates` -- Bicep-based AKS deployment as an alternative to Terraform.
- `azure-vms` -- Understanding VM sizes for node pool selection.
infrastructure/cloud-azure/azure-functions/SKILL.md
+383 -17
View File
@@ -9,46 +9,412 @@ metadata:
# Azure Functions
Build serverless applications with Azure Functions.
Build and deploy serverless applications with Azure Functions. Covers function app creation, trigger and binding configuration, deployment strategies, real code examples in Python and Node.js, and production best practices.
## Create Function App
## When to Use
- You need event-driven compute that scales automatically to zero.
- You are building APIs, webhooks, or background processing pipelines.
- You want per-execution billing without managing servers.
- You need to respond to Azure service events (Blob Storage, Service Bus, Cosmos DB changes).
- You are implementing lightweight microservices or scheduled tasks.
## Prerequisites
```bash
# Install Azure CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
# Install Azure Functions Core Tools v4
npm install -g azure-functions-core-tools@4
# Verify installation
func --version
# Login
az login
az account set --subscription "my-subscription-id"
# Create supporting resources
az group create --name functions-rg --location eastus
az storage account create \
--name myfuncstorageacct \
--resource-group functions-rg \
--location eastus \
--sku Standard_LRS
```
## Function App Creation
### Consumption Plan (Pay-per-execution)
```bash
# Python function app on Consumption plan
az functionapp create \
--resource-group mygroup \
--resource-group functions-rg \
--consumption-plan-location eastus \
--runtime python \
--runtime-version 3.11 \
--functions-version 4 \
--name myfunctionapp \
--storage-account mystorageaccount
--name myapp-func \
--storage-account myfuncstorageacct \
--os-type Linux
# Node.js function app
az functionapp create \
--resource-group functions-rg \
--consumption-plan-location eastus \
--runtime node \
--runtime-version 20 \
--functions-version 4 \
--name myapp-node-func \
--storage-account myfuncstorageacct \
--os-type Linux
```
## Function Code
### Premium Plan (VNet integration, no cold start)
```bash
# Create Premium plan
az functionapp plan create \
--resource-group functions-rg \
--name myapp-premium-plan \
--location eastus \
--sku EP1 \
--is-linux true
# Create function app on Premium plan
az functionapp create \
--resource-group functions-rg \
--plan myapp-premium-plan \
--runtime python \
--runtime-version 3.11 \
--functions-version 4 \
--name myapp-premium-func \
--storage-account myfuncstorageacct
```
## Trigger and Binding Examples
### HTTP Trigger -- Python
```python
# function_app.py (v2 programming model)
import azure.functions as func
import json
import logging
def main(req: func.HttpRequest) -> func.HttpResponse:
return func.HttpResponse("Hello, World!")
app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION)
@app.route(route="users/{userId}", methods=["GET"])
def get_user(req: func.HttpRequest) -> func.HttpResponse:
user_id = req.route_params.get("userId")
logging.info(f"Fetching user: {user_id}")
if not user_id:
return func.HttpResponse(
json.dumps({"error": "userId is required"}),
status_code=400,
mimetype="application/json"
)
user = {"id": user_id, "name": "Jane Doe", "email": "jane@example.com"}
return func.HttpResponse(
json.dumps(user),
status_code=200,
mimetype="application/json"
)
@app.route(route="users", methods=["POST"])
def create_user(req: func.HttpRequest) -> func.HttpResponse:
try:
body = req.get_json()
except ValueError:
return func.HttpResponse(
json.dumps({"error": "Invalid JSON"}),
status_code=400,
mimetype="application/json"
)
logging.info(f"Creating user: {body.get('name')}")
return func.HttpResponse(
json.dumps({"id": "new-id", **body}),
status_code=201,
mimetype="application/json"
)
```
### HTTP Trigger -- Node.js
```javascript
// src/functions/httpTrigger.js (v4 programming model)
const { app } = require("@azure/functions");
app.http("getUser", {
methods: ["GET"],
authLevel: "function",
route: "users/{userId}",
handler: async (request, context) => {
const userId = request.params.userId;
context.log(`Fetching user: ${userId}`);
if (!userId) {
return { status: 400, jsonBody: { error: "userId is required" } };
}
const user = { id: userId, name: "Jane Doe", email: "jane@example.com" };
return { status: 200, jsonBody: user };
},
});
app.http("createUser", {
methods: ["POST"],
authLevel: "function",
route: "users",
handler: async (request, context) => {
const body = await request.json();
context.log(`Creating user: ${body.name}`);
return { status: 201, jsonBody: { id: "new-id", ...body } };
},
});
```
### Blob Trigger -- Python
```python
@app.blob_trigger(arg_name="blob", path="uploads/{name}",
connection="AzureWebJobsStorage")
def process_upload(blob: func.InputStream):
logging.info(f"Processing blob: {blob.name}, Size: {blob.length} bytes")
content = blob.read()
# Process file content here
```
### Timer Trigger -- Python
```python
@app.timer_trigger(schedule="0 */5 * * * *", arg_name="timer",
run_on_startup=False)
def cleanup_job(timer: func.TimerRequest):
if timer.past_due:
logging.warning("Timer is past due")
logging.info("Running scheduled cleanup")
# Cleanup logic here
```
### Service Bus Trigger -- Python
```python
@app.service_bus_queue_trigger(arg_name="msg", queue_name="orders",
connection="ServiceBusConnection")
@app.cosmos_db_output(arg_name="doc", database_name="mydb",
container_name="processed-orders",
connection="CosmosDBConnection")
def process_order(msg: func.ServiceBusMessage, doc: func.Out[func.Document]):
order = json.loads(msg.get_body().decode("utf-8"))
logging.info(f"Processing order: {order['id']}")
processed = {
"id": order["id"],
"status": "processed",
"items": order["items"],
"total": sum(item["price"] for item in order["items"])
}
doc.set(func.Document.from_dict(processed))
```
### Cosmos DB Change Feed Trigger -- Python
```python
@app.cosmos_db_trigger_v3(arg_name="documents", database_name="mydb",
container_name="orders",
connection="CosmosDBConnection",
lease_container_name="leases",
create_lease_container_if_not_exists=True)
def on_order_change(documents: func.DocumentList):
for doc in documents:
logging.info(f"Document changed: {doc['id']}")
```
## Local Development
```bash
# Initialize a new Python function project
func init MyFunctionProject --python
cd MyFunctionProject
# Create a new function from template
func new --name HttpExample --template "HTTP trigger" --authlevel function
# Run locally
func start
# Run locally with specific port
func start --port 7072
# Test locally
curl http://localhost:7071/api/HttpExample?name=World
```
## Deployment
```bash
# Deploy using Core Tools
func azure functionapp publish myfunctionapp
func azure functionapp publish myapp-func
# Deploy using ZIP
# Deploy with build step for Python
func azure functionapp publish myapp-func --build remote
# Deploy using ZIP package
zip -r function.zip . -x ".git/*" ".venv/*" "__pycache__/*"
az functionapp deployment source config-zip \
--resource-group mygroup \
--name myfunctionapp \
--resource-group functions-rg \
--name myapp-func \
--src function.zip
# Deploy via CI/CD with GitHub Actions
az functionapp deployment github-actions add \
--resource-group functions-rg \
--name myapp-func \
--repo "myorg/myrepo" \
--branch main \
--runtime python \
--login-with-github
```
## Best Practices
## Deployment Slots
- Use consumption plan for variable workloads
- Implement Durable Functions for orchestration
- Use managed identity for authentication
- Monitor with Application Insights
```bash
# Create a staging slot
az functionapp deployment slot create \
--resource-group functions-rg \
--name myapp-func \
--slot staging
# Deploy to staging slot
func azure functionapp publish myapp-func --slot staging
# Test staging slot
curl https://myapp-func-staging.azurewebsites.net/api/health
# Swap staging to production
az functionapp deployment slot swap \
--resource-group functions-rg \
--name myapp-func \
--slot staging \
--target-slot production
# Roll back by swapping again
az functionapp deployment slot swap \
--resource-group functions-rg \
--name myapp-func \
--slot staging \
--target-slot production
```
## Application Settings and Security
```bash
# Set application settings
az functionapp config appsettings set \
--resource-group functions-rg \
--name myapp-func \
--settings \
ServiceBusConnection="Endpoint=sb://..." \
CosmosDBConnection="AccountEndpoint=https://..." \
CUSTOM_SETTING="my-value"
# Set settings as slot-specific
az functionapp config appsettings set \
--resource-group functions-rg \
--name myapp-func \
--slot-settings \
ENVIRONMENT="staging"
# Enable managed identity
az functionapp identity assign \
--resource-group functions-rg \
--name myapp-func
# Configure CORS
az functionapp cors add \
--resource-group functions-rg \
--name myapp-func \
--allowed-origins "https://myapp.example.com"
# Set minimum TLS version
az functionapp config set \
--resource-group functions-rg \
--name myapp-func \
--min-tls-version 1.2
# Enable Application Insights
az functionapp config appsettings set \
--resource-group functions-rg \
--name myapp-func \
--settings APPINSIGHTS_INSTRUMENTATIONKEY="your-key"
```
## Terraform Configuration
```hcl
resource "azurerm_service_plan" "functions" {
name = "myapp-func-plan"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
os_type = "Linux"
sku_name = "Y1" # Consumption plan
}
resource "azurerm_linux_function_app" "main" {
name = "myapp-func"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
service_plan_id = azurerm_service_plan.functions.id
storage_account_name = azurerm_storage_account.func.name
storage_account_access_key = azurerm_storage_account.func.primary_access_key
identity {
type = "SystemAssigned"
}
site_config {
application_stack {
python_version = "3.11"
}
cors {
allowed_origins = ["https://myapp.example.com"]
}
}
app_settings = {
FUNCTIONS_WORKER_RUNTIME = "python"
WEBSITE_RUN_FROM_PACKAGE = "1"
APPINSIGHTS_INSTRUMENTATIONKEY = azurerm_application_insights.main.instrumentation_key
}
tags = var.tags
}
```
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| Cold start latency > 10s | Consumption plan cold start | Use Premium plan (EP1+) or enable `WEBSITE_RUN_FROM_PACKAGE=1` |
| Function not triggering | Connection string misconfigured | Check `az functionapp config appsettings list` for correct binding values |
| `ModuleNotFoundError` in Python | Dependencies not installed during deploy | Use `--build remote` flag or include `requirements.txt` in package |
| HTTP 401 Unauthorized | Auth level mismatch or missing function key | Verify auth level in code matches expectations; pass `x-functions-key` header |
| Blob trigger not firing | Storage account connection wrong | Verify `AzureWebJobsStorage` points to the correct account |
| Timer trigger runs twice | Multiple instances on Premium plan | Set `WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT=1` or use singleton lock |
| Deployment slot swap fails | Slot settings not configured | Ensure slot-specific settings are marked with `--slot-settings` |
| Out of memory errors | Large payloads or memory leaks | Stream data instead of loading entirely; increase plan tier |
## Related Skills
- `azure-networking` -- VNet integration for Premium plan functions accessing private resources.
- `azure-sql` -- Database connections from function bindings.
- `terraform-azure` -- Infrastructure as Code for function app provisioning.
- `arm-templates` -- Bicep-based function app deployment.
infrastructure/cloud-azure/azure-networking/SKILL.md
+525 -31
View File
@@ -9,52 +9,546 @@ metadata:
# Azure Networking
Design and implement Azure network infrastructure.
Design and implement Azure network infrastructure including VNets, subnets, NSGs, VNet peering, private endpoints, Azure Firewall, and Application Gateway. Covers both az CLI commands and Terraform configurations for production hub-spoke topologies.
## Create VNet
## When to Use
- You are designing the network foundation for Azure workloads.
- You need to isolate environments with VNets and NSGs.
- You are connecting on-premises networks to Azure via VPN or ExpressRoute.
- You need private connectivity to PaaS services via private endpoints.
- You are implementing centralized egress filtering with Azure Firewall.
- You need to set up load balancing or application-layer routing with Application Gateway.
## Prerequisites
```bash
az network vnet create \
--resource-group mygroup \
--name myvnet \
--address-prefix 10.0.0.0/16 \
--subnet-name default \
--subnet-prefix 10.0.1.0/24
# Install Azure CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
# Login and set subscription
az login
az account set --subscription "my-subscription-id"
# Register required providers
az provider register --namespace Microsoft.Network
# Create resource group
az group create --name networking-rg --location eastus
```
## Network Security Group
## VNet and Subnet Creation
### Hub VNet
```bash
az network nsg create \
--resource-group mygroup \
--name mynsg
# Create hub VNet for shared services
az network vnet create \
--resource-group networking-rg \
--name hub-vnet \
--address-prefix 10.0.0.0/16 \
--location eastus \
--tags environment=prod role=hub
# Add subnets to hub
az network vnet subnet create \
--resource-group networking-rg \
--vnet-name hub-vnet \
--name AzureFirewallSubnet \
--address-prefix 10.0.1.0/26
az network vnet subnet create \
--resource-group networking-rg \
--vnet-name hub-vnet \
--name GatewaySubnet \
--address-prefix 10.0.2.0/27
az network vnet subnet create \
--resource-group networking-rg \
--vnet-name hub-vnet \
--name SharedServicesSubnet \
--address-prefix 10.0.3.0/24
az network vnet subnet create \
--resource-group networking-rg \
--vnet-name hub-vnet \
--name AzureBastionSubnet \
--address-prefix 10.0.4.0/26
```
### Spoke VNet
```bash
# Create spoke VNet for application workloads
az network vnet create \
--resource-group networking-rg \
--name spoke-prod-vnet \
--address-prefix 10.1.0.0/16 \
--location eastus \
--tags environment=prod role=spoke
az network vnet subnet create \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--name web-subnet \
--address-prefix 10.1.1.0/24
az network vnet subnet create \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--name app-subnet \
--address-prefix 10.1.2.0/24
az network vnet subnet create \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--name data-subnet \
--address-prefix 10.1.3.0/24 \
--private-endpoint-network-policies Enabled
# List all subnets in a VNet
az network vnet subnet list \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--output table
```
## Network Security Groups
```bash
# Create NSG for web tier
az network nsg create \
--resource-group networking-rg \
--name web-nsg \
--tags tier=web
# Allow HTTPS from internet
az network nsg rule create \
--resource-group mygroup \
--nsg-name mynsg \
--resource-group networking-rg \
--nsg-name web-nsg \
--name AllowHTTPS \
--priority 100 \
--destination-port-ranges 443 \
--access Allow
--direction Inbound \
--access Allow \
--protocol Tcp \
--source-address-prefixes Internet \
--destination-port-ranges 443
# Allow HTTP for redirect
az network nsg rule create \
--resource-group networking-rg \
--nsg-name web-nsg \
--name AllowHTTP \
--priority 110 \
--direction Inbound \
--access Allow \
--protocol Tcp \
--source-address-prefixes Internet \
--destination-port-ranges 80
# Deny all other inbound traffic
az network nsg rule create \
--resource-group networking-rg \
--nsg-name web-nsg \
--name DenyAllInbound \
--priority 4096 \
--direction Inbound \
--access Deny \
--protocol '*' \
--source-address-prefixes '*' \
--destination-port-ranges '*'
# Create NSG for app tier -- only allow from web subnet
az network nsg create \
--resource-group networking-rg \
--name app-nsg
az network nsg rule create \
--resource-group networking-rg \
--nsg-name app-nsg \
--name AllowFromWeb \
--priority 100 \
--direction Inbound \
--access Allow \
--protocol Tcp \
--source-address-prefixes 10.1.1.0/24 \
--destination-port-ranges 8080
# Create NSG for data tier -- only allow from app subnet
az network nsg create \
--resource-group networking-rg \
--name data-nsg
az network nsg rule create \
--resource-group networking-rg \
--nsg-name data-nsg \
--name AllowSQLFromApp \
--priority 100 \
--direction Inbound \
--access Allow \
--protocol Tcp \
--source-address-prefixes 10.1.2.0/24 \
--destination-port-ranges 1433
# Associate NSG with subnet
az network vnet subnet update \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--name web-subnet \
--network-security-group web-nsg
az network vnet subnet update \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--name app-subnet \
--network-security-group app-nsg
az network vnet subnet update \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--name data-subnet \
--network-security-group data-nsg
# View effective NSG rules
az network nic list-effective-nsg \
--resource-group networking-rg \
--name myvm-nic \
--output table
```
## Private Endpoint
## VNet Peering
```bash
az network private-endpoint create \
--resource-group mygroup \
--name myendpoint \
--vnet-name myvnet \
--subnet default \
--private-connection-resource-id /subscriptions/.../sql/... \
--group-id sqlServer \
--connection-name myconnection
# Peer hub to spoke
az network vnet peering create \
--resource-group networking-rg \
--name hub-to-spoke-prod \
--vnet-name hub-vnet \
--remote-vnet spoke-prod-vnet \
--allow-vnet-access \
--allow-forwarded-traffic \
--allow-gateway-transit
# Peer spoke to hub
az network vnet peering create \
--resource-group networking-rg \
--name spoke-prod-to-hub \
--vnet-name spoke-prod-vnet \
--remote-vnet hub-vnet \
--allow-vnet-access \
--allow-forwarded-traffic \
--use-remote-gateways false
# Verify peering status
az network vnet peering list \
--resource-group networking-rg \
--vnet-name hub-vnet \
--output table
```
## Best Practices
## Private Endpoints
- Implement hub-spoke topology
- Use NSGs and Azure Firewall
- Enable DDoS protection
- Use private endpoints
- Implement VNet peering
```bash
# Create private endpoint for Azure SQL
az network private-endpoint create \
--resource-group networking-rg \
--name sql-private-endpoint \
--vnet-name spoke-prod-vnet \
--subnet data-subnet \
--private-connection-resource-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Sql/servers/myserver" \
--group-id sqlServer \
--connection-name sql-connection
# Create private DNS zone for SQL
az network private-dns zone create \
--resource-group networking-rg \
--name privatelink.database.windows.net
# Link DNS zone to VNet
az network private-dns link vnet create \
--resource-group networking-rg \
--zone-name privatelink.database.windows.net \
--name spoke-dns-link \
--virtual-network spoke-prod-vnet \
--registration-enabled false
# Create DNS record for the private endpoint
az network private-endpoint dns-zone-group create \
--resource-group networking-rg \
--endpoint-name sql-private-endpoint \
--name sql-dns-group \
--private-dns-zone privatelink.database.windows.net \
--zone-name privatelink.database.windows.net
# Create private endpoint for Storage Account
az network private-endpoint create \
--resource-group networking-rg \
--name storage-private-endpoint \
--vnet-name spoke-prod-vnet \
--subnet data-subnet \
--private-connection-resource-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Storage/storageAccounts/mystorageacct" \
--group-id blob \
--connection-name storage-blob-connection
# Create private endpoint for Key Vault
az network private-endpoint create \
--resource-group networking-rg \
--name kv-private-endpoint \
--vnet-name spoke-prod-vnet \
--subnet app-subnet \
--private-connection-resource-id "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.KeyVault/vaults/myvault" \
--group-id vault \
--connection-name kv-connection
```
## Azure Firewall
```bash
# Create public IP for firewall
az network public-ip create \
--resource-group networking-rg \
--name fw-public-ip \
--sku Standard \
--allocation-method Static
# Create Azure Firewall
az network firewall create \
--resource-group networking-rg \
--name hub-firewall \
--location eastus \
--sku AZFW_VNet \
--tier Standard
# Configure firewall IP
az network firewall ip-config create \
--resource-group networking-rg \
--firewall-name hub-firewall \
--name fw-ipconfig \
--public-ip-address fw-public-ip \
--vnet-name hub-vnet
# Get firewall private IP for route tables
FW_PRIVATE_IP=$(az network firewall show \
--resource-group networking-rg \
--name hub-firewall \
--query "ipConfigurations[0].privateIpAddress" \
--output tsv)
# Create application rule allowing web traffic
az network firewall application-rule create \
--resource-group networking-rg \
--firewall-name hub-firewall \
--collection-name AllowWeb \
--name AllowGoogle \
--protocols Https=443 Http=80 \
--source-addresses 10.1.0.0/16 \
--target-fqdns "*.google.com" "*.microsoft.com" \
--action Allow \
--priority 100
# Create network rule for DNS
az network firewall network-rule create \
--resource-group networking-rg \
--firewall-name hub-firewall \
--collection-name AllowDNS \
--name AllowDNS \
--protocols UDP \
--source-addresses 10.1.0.0/16 \
--destination-addresses 168.63.129.16 \
--destination-ports 53 \
--action Allow \
--priority 200
# Create route table to send traffic through firewall
az network route-table create \
--resource-group networking-rg \
--name spoke-route-table
az network route-table route create \
--resource-group networking-rg \
--route-table-name spoke-route-table \
--name default-to-firewall \
--address-prefix 0.0.0.0/0 \
--next-hop-type VirtualAppliance \
--next-hop-ip-address "$FW_PRIVATE_IP"
# Associate route table with spoke subnet
az network vnet subnet update \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--name app-subnet \
--route-table spoke-route-table
```
## Application Gateway with WAF
```bash
# Create public IP
az network public-ip create \
--resource-group networking-rg \
--name appgw-public-ip \
--sku Standard \
--allocation-method Static
# Create Application Gateway subnet
az network vnet subnet create \
--resource-group networking-rg \
--vnet-name spoke-prod-vnet \
--name AppGatewaySubnet \
--address-prefix 10.1.10.0/24
# Create Application Gateway with WAF v2
az network application-gateway create \
--resource-group networking-rg \
--name myapp-appgw \
--location eastus \
--sku WAF_v2 \
--capacity 2 \
--vnet-name spoke-prod-vnet \
--subnet AppGatewaySubnet \
--public-ip-address appgw-public-ip \
--http-settings-port 80 \
--http-settings-protocol Http \
--frontend-port 443 \
--servers 10.1.2.4 10.1.2.5
# Enable WAF policy
az network application-gateway waf-policy create \
--resource-group networking-rg \
--name myapp-waf-policy
az network application-gateway waf-policy managed-rule rule-set add \
--resource-group networking-rg \
--policy-name myapp-waf-policy \
--type OWASP \
--version 3.2
```
## Terraform Configuration
```hcl
resource "azurerm_virtual_network" "hub" {
name = "hub-vnet"
location = azurerm_resource_group.networking.location
resource_group_name = azurerm_resource_group.networking.name
address_space = ["10.0.0.0/16"]
tags = var.tags
}
resource "azurerm_subnet" "firewall" {
name = "AzureFirewallSubnet"
resource_group_name = azurerm_resource_group.networking.name
virtual_network_name = azurerm_virtual_network.hub.name
address_prefixes = ["10.0.1.0/26"]
}
resource "azurerm_virtual_network" "spoke" {
name = "spoke-prod-vnet"
location = azurerm_resource_group.networking.location
resource_group_name = azurerm_resource_group.networking.name
address_space = ["10.1.0.0/16"]
tags = var.tags
}
resource "azurerm_subnet" "web" {
name = "web-subnet"
resource_group_name = azurerm_resource_group.networking.name
virtual_network_name = azurerm_virtual_network.spoke.name
address_prefixes = ["10.1.1.0/24"]
}
resource "azurerm_network_security_group" "web" {
name = "web-nsg"
location = azurerm_resource_group.networking.location
resource_group_name = azurerm_resource_group.networking.name
security_rule {
name = "AllowHTTPS"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "443"
source_address_prefix = "Internet"
destination_address_prefix = "*"
}
security_rule {
name = "DenyAllInbound"
priority = 4096
direction = "Inbound"
access = "Deny"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
tags = var.tags
}
resource "azurerm_subnet_network_security_group_association" "web" {
subnet_id = azurerm_subnet.web.id
network_security_group_id = azurerm_network_security_group.web.id
}
resource "azurerm_virtual_network_peering" "hub_to_spoke" {
name = "hub-to-spoke"
resource_group_name = azurerm_resource_group.networking.name
virtual_network_name = azurerm_virtual_network.hub.name
remote_virtual_network_id = azurerm_virtual_network.spoke.id
allow_forwarded_traffic = true
allow_gateway_transit = true
}
resource "azurerm_virtual_network_peering" "spoke_to_hub" {
name = "spoke-to-hub"
resource_group_name = azurerm_resource_group.networking.name
virtual_network_name = azurerm_virtual_network.spoke.name
remote_virtual_network_id = azurerm_virtual_network.hub.id
allow_forwarded_traffic = true
use_remote_gateways = false
}
resource "azurerm_private_endpoint" "sql" {
name = "sql-private-endpoint"
location = azurerm_resource_group.networking.location
resource_group_name = azurerm_resource_group.networking.name
subnet_id = azurerm_subnet.data.id
private_service_connection {
name = "sql-connection"
private_connection_resource_id = azurerm_mssql_server.main.id
subresource_names = ["sqlServer"]
is_manual_connection = false
}
private_dns_zone_group {
name = "sql-dns-group"
private_dns_zone_ids = [azurerm_private_dns_zone.sql.id]
}
}
```
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| VMs cannot reach the internet | NSG blocking outbound or missing route | Check NSG rules with `az network nic list-effective-nsg`; verify route table |
| VNet peering shows `Disconnected` | Peering created in one direction only | Create peering from both sides (hub-to-spoke AND spoke-to-hub) |
| Private endpoint DNS not resolving | Private DNS zone not linked to VNet | Link DNS zone with `az network private-dns link vnet create` |
| NSG rule not taking effect | Higher-priority rule overriding | List rules with `az network nsg rule list` and check priority ordering |
| Application Gateway health probes failing | Backend pool servers unreachable | Verify NSG allows traffic from the AppGateway subnet |
| Azure Firewall blocking legitimate traffic | Missing application or network rule | Check firewall logs in Log Analytics; add appropriate rule |
| Cross-VNet communication failing | Peering not configured or route missing | Verify peering status and that `allow-vnet-access` is enabled |
| High latency between regions | Traffic routing through unexpected path | Use `az network watcher next-hop` to diagnose routing |
## Related Skills
- `azure-vms` -- VM network interface and NSG configuration.
- `azure-aks` -- AKS VNet integration with Azure CNI.
- `azure-sql` -- Private endpoint configuration for database access.
- `terraform-azure` -- Network infrastructure provisioning with Terraform.
- `azure-functions` -- VNet integration for Premium plan functions.
infrastructure/cloud-azure/azure-sql/SKILL.md
+466 -27
View File
@@ -9,50 +9,489 @@ metadata:
# Azure SQL
Deploy managed databases on Azure.
Deploy and manage Azure SQL Database, Elastic Pools, and Cosmos DB. Covers server provisioning, firewall rules, geo-replication, backup strategies, performance tuning, security hardening, and Terraform configurations.
## Create SQL Database
## When to Use
- You need a fully managed relational database on Azure.
- Your application requires geo-replication for disaster recovery.
- You need elastic scaling across multiple databases with Elastic Pools.
- You are migrating on-premises SQL Server workloads to the cloud.
- You need a globally distributed NoSQL database (Cosmos DB).
## Prerequisites
```bash
# Create server
az sql server create \
--name myserver \
--resource-group mygroup \
--admin-user sqladmin \
--admin-password SecureP@ss123
# Install Azure CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
# Create database
az sql db create \
--resource-group mygroup \
--server myserver \
--name mydb \
--service-objective S1
# Login and set subscription
az login
az account set --subscription "my-subscription-id"
# Create resource group
az group create --name database-rg --location eastus
```
## Firewall Rules
## SQL Server and Database Creation
### Create SQL Server
```bash
# Create logical SQL server
az sql server create \
--resource-group database-rg \
--name myapp-sqlserver \
--location eastus \
--admin-user sqladmin \
--admin-password 'S3cur3P@ssw0rd!' \
--enable-public-network false \
--minimal-tls-version 1.2
# Enable Azure AD authentication
az sql server ad-admin create \
--resource-group database-rg \
--server-name myapp-sqlserver \
--display-name "SQL Admins" \
--object-id "{aad-group-object-id}"
# Enable Azure AD only authentication (disable SQL auth)
az sql server ad-only-auth enable \
--resource-group database-rg \
--name myapp-sqlserver
```
### Create Databases
```bash
# Create General Purpose database
az sql db create \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-db \
--edition GeneralPurpose \
--compute-model Serverless \
--auto-pause-delay 60 \
--min-capacity 0.5 \
--max-size 32GB \
--backup-storage-redundancy Geo \
--zone-redundant false
# Create Business Critical database for production
az sql db create \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-prod-db \
--edition BusinessCritical \
--service-objective BC_Gen5_4 \
--max-size 256GB \
--backup-storage-redundancy Geo \
--zone-redundant true \
--read-scale Enabled
# Create Hyperscale database for large workloads
az sql db create \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-analytics-db \
--edition Hyperscale \
--service-objective HS_Gen5_4 \
--ha-replicas 2
# List databases on server
az sql db list \
--resource-group database-rg \
--server myapp-sqlserver \
--output table
```
### Elastic Pools
```bash
# Create elastic pool
az sql elastic-pool create \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-pool \
--edition GeneralPurpose \
--capacity 4 \
--db-max-capacity 2 \
--db-min-capacity 0.25 \
--max-size 256GB \
--zone-redundant false
# Move database into elastic pool
az sql db update \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-db \
--elastic-pool myapp-pool
# Monitor elastic pool usage
az sql elastic-pool list-dbs \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-pool \
--output table
```
## Firewall Rules and Network Security
```bash
# Allow Azure services
az sql server firewall-rule create \
--resource-group mygroup \
--server myserver \
--name AllowAzure \
--resource-group database-rg \
--server myapp-sqlserver \
--name AllowAzureServices \
--start-ip-address 0.0.0.0 \
--end-ip-address 0.0.0.0
# Allow specific IP range (office network)
az sql server firewall-rule create \
--resource-group database-rg \
--server myapp-sqlserver \
--name AllowOffice \
--start-ip-address 203.0.113.0 \
--end-ip-address 203.0.113.255
# Allow your current client IP
az sql server firewall-rule create \
--resource-group database-rg \
--server myapp-sqlserver \
--name AllowMyIP \
--start-ip-address "$(curl -s ifconfig.me)" \
--end-ip-address "$(curl -s ifconfig.me)"
# Create VNet rule for subnet access
az sql server vnet-rule create \
--resource-group database-rg \
--server myapp-sqlserver \
--name AllowAppSubnet \
--vnet-name spoke-prod-vnet \
--subnet app-subnet
# List firewall rules
az sql server firewall-rule list \
--resource-group database-rg \
--server myapp-sqlserver \
--output table
# Remove a firewall rule
az sql server firewall-rule delete \
--resource-group database-rg \
--server myapp-sqlserver \
--name AllowMyIP
```
## Geo-Replication and Failover
```bash
# Create failover group with secondary server
az sql server create \
--resource-group database-rg \
--name myapp-sqlserver-secondary \
--location westus \
--admin-user sqladmin \
--admin-password 'S3cur3P@ssw0rd!'
az sql failover-group create \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-failover-group \
--partner-server myapp-sqlserver-secondary \
--partner-resource-group database-rg \
--failover-policy Automatic \
--grace-period 1 \
--add-db myapp-prod-db
# Check failover group status
az sql failover-group show \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-failover-group \
--output table
# Manual failover (for testing or planned maintenance)
az sql failover-group set-primary \
--resource-group database-rg \
--server myapp-sqlserver-secondary \
--name myapp-failover-group
# Create active geo-replication (without failover group)
az sql db replica create \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-db \
--partner-server myapp-sqlserver-secondary \
--partner-resource-group database-rg
```
## Backup and Restore
```bash
# Configure short-term retention (1-35 days)
az sql db str-policy set \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-prod-db \
--retention-days 14 \
--diffbackup-hours 12
# Configure long-term retention
az sql db ltr-policy set \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-prod-db \
--weekly-retention P4W \
--monthly-retention P12M \
--yearly-retention P5Y \
--week-of-year 1
# Restore database to a point in time
az sql db restore \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-db-restored \
--dest-name myapp-db-restored \
--time "2026-03-23T10:00:00Z"
# Restore from long-term backup
az sql db ltr-backup list \
--resource-group database-rg \
--server myapp-sqlserver \
--database myapp-prod-db \
--output table
# Export database to bacpac
az sql db export \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-db \
--admin-user sqladmin \
--admin-password 'S3cur3P@ssw0rd!' \
--storage-key-type StorageAccessKey \
--storage-key "{storage-account-key}" \
--storage-uri "https://mystorageacct.blob.core.windows.net/backups/myapp-db.bacpac"
```
## Performance Tuning
```bash
# Enable automatic tuning
az sql db update \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-prod-db \
--set tags.autoTuning=enabled
# Check database performance recommendations
az sql db advisor list \
--resource-group database-rg \
--server myapp-sqlserver \
--database myapp-prod-db \
--output table
# Scale database tier
az sql db update \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-prod-db \
--service-objective BC_Gen5_8
# Enable Query Store (via SQL)
# sqlcmd -S myapp-sqlserver.database.windows.net -d myapp-prod-db -Q "ALTER DATABASE [myapp-prod-db] SET QUERY_STORE = ON"
# View DTU/vCore usage metrics
az monitor metrics list \
--resource "/subscriptions/{sub}/resourceGroups/database-rg/providers/Microsoft.Sql/servers/myapp-sqlserver/databases/myapp-prod-db" \
--metric "cpu_percent" "dtu_consumption_percent" "storage_percent" \
--interval PT1H \
--output table
```
## Security Hardening
```bash
# Enable Advanced Threat Protection
az sql db threat-policy update \
--resource-group database-rg \
--server myapp-sqlserver \
--name myapp-prod-db \
--state Enabled \
--email-addresses security@example.com \
--email-account-admins true
# Enable auditing to storage
az sql server audit-policy update \
--resource-group database-rg \
--name myapp-sqlserver \
--state Enabled \
--storage-account mystorageacct \
--retention-days 90
# Enable auditing to Log Analytics
az sql server audit-policy update \
--resource-group database-rg \
--name myapp-sqlserver \
--state Enabled \
--lats Enabled \
--lawri "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.OperationalInsights/workspaces/{workspace}"
# Enable Transparent Data Encryption (enabled by default)
az sql db tde set \
--resource-group database-rg \
--server myapp-sqlserver \
--database myapp-prod-db \
--status Enabled
# Enable vulnerability assessment
az sql vm update \
--resource-group database-rg \
--name myapp-sqlserver
```
## Cosmos DB
```bash
# Create Cosmos DB account with SQL API
az cosmosdb create \
--name mycosmosdb \
--resource-group mygroup \
--default-consistency-level Session
--resource-group database-rg \
--name myapp-cosmos \
--default-consistency-level Session \
--locations regionName=eastus failoverPriority=0 isZoneRedundant=true \
--locations regionName=westus failoverPriority=1 isZoneRedundant=false \
--enable-automatic-failover true \
--enable-multiple-write-locations false
# Create database
az cosmosdb sql database create \
--resource-group database-rg \
--account-name myapp-cosmos \
--name myappdb \
--throughput 400
# Create container with partition key
az cosmosdb sql container create \
--resource-group database-rg \
--account-name myapp-cosmos \
--database-name myappdb \
--name orders \
--partition-key-path "/customerId" \
--throughput 400 \
--idx @indexing-policy.json
# Enable autoscale throughput
az cosmosdb sql container throughput update \
--resource-group database-rg \
--account-name myapp-cosmos \
--database-name myappdb \
--name orders \
--max-throughput 4000
```
## Best Practices
## Terraform Configuration
- Enable transparent data encryption
- Use Azure AD authentication
- Implement geo-replication
- Configure automated backups
- Use private endpoints
```hcl
resource "azurerm_mssql_server" "main" {
name = "myapp-sqlserver"
resource_group_name = azurerm_resource_group.main.name
location = azurerm_resource_group.main.location
version = "12.0"
administrator_login = "sqladmin"
administrator_login_password = var.sql_admin_password
minimum_tls_version = "1.2"
public_network_access_enabled = false
azuread_administrator {
login_username = "SQL Admins"
object_id = var.sql_admin_aad_group_id
}
tags = var.tags
}
resource "azurerm_mssql_database" "main" {
name = "myapp-prod-db"
server_id = azurerm_mssql_server.main.id
collation = "SQL_Latin1_General_CP1_CI_AS"
license_type = "LicenseIncluded"
sku_name = "BC_Gen5_4"
max_size_gb = 256
zone_redundant = true
read_scale = true
short_term_retention_policy {
retention_days = 14
backup_interval_in_hours = 12
}
long_term_retention_policy {
weekly_retention = "P4W"
monthly_retention = "P12M"
yearly_retention = "P5Y"
week_of_year = 1
}
threat_detection_policy {
state = "Enabled"
email_addresses = ["security@example.com"]
email_account_admins = "Enabled"
retention_days = 90
storage_endpoint = azurerm_storage_account.audit.primary_blob_endpoint
storage_account_access_key = azurerm_storage_account.audit.primary_access_key
}
tags = var.tags
}
resource "azurerm_mssql_failover_group" "main" {
name = "myapp-failover-group"
server_id = azurerm_mssql_server.main.id
databases = [azurerm_mssql_database.main.id]
partner_server {
id = azurerm_mssql_server.secondary.id
}
read_write_endpoint_failover_policy {
mode = "Automatic"
grace_minutes = 60
}
tags = var.tags
}
resource "azurerm_private_endpoint" "sql" {
name = "sql-private-endpoint"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
subnet_id = azurerm_subnet.data.id
private_service_connection {
name = "sql-connection"
private_connection_resource_id = azurerm_mssql_server.main.id
subresource_names = ["sqlServer"]
is_manual_connection = false
}
}
```
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| Cannot connect to SQL server | Firewall rule missing or public access disabled | Add client IP with `az sql server firewall-rule create` or use private endpoint |
| Login failed for user | Incorrect credentials or Azure AD not configured | Verify admin credentials; enable Azure AD auth on the server |
| Database DTU at 100% | Under-provisioned tier or inefficient queries | Scale up service objective; review Query Store for expensive queries |
| Geo-replication lag is high | Large transaction volumes or network latency | Monitor with `sys.dm_geo_replication_link_status`; consider Hyperscale |
| Point-in-time restore fails | Requested time is outside retention window | Check retention policy; use long-term backups for older data |
| Elastic pool running out of eDTUs | Too many active databases in pool | Increase pool capacity or move heavy databases to dedicated tier |
| TDE key rotation failure | Key Vault access policy missing | Grant SQL server managed identity GET, WRAP, UNWRAP permissions |
| Connection timeout from app | Network path blocked or DNS issue | Use `az network watcher test-connectivity`; verify private DNS resolution |
## Related Skills
- `azure-networking` -- Private endpoints and VNet rules for SQL access.
- `azure-functions` -- SQL bindings for serverless data access.
- `terraform-azure` -- Terraform-based SQL infrastructure provisioning.
- `arm-templates` -- Bicep templates for SQL deployments.
infrastructure/cloud-azure/azure-vms/SKILL.md
+481 -19
View File
@@ -9,37 +9,499 @@ metadata:
# Azure Virtual Machines
Deploy and manage Azure VMs and scale sets.
Deploy and manage Azure VMs, availability sets, scale sets, custom images, and managed disks. Covers VM creation, sizing, disk management, auto-scaling, and Terraform configurations for production environments.
## Create VM
## When to Use
- You need full control over the operating system and runtime environment.
- Your application requires specific OS configurations or kernel modules.
- You are running legacy applications that cannot be containerized.
- You need GPU-accelerated compute for ML training or rendering.
- You need high-availability compute with availability zones or scale sets.
## Prerequisites
```bash
# Install Azure CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
# Login and set subscription
az login
az account set --subscription "my-subscription-id"
# Create resource group
az group create --name compute-rg --location eastus
# List available VM sizes in a region
az vm list-sizes --location eastus --output table
# List available VM images
az vm image list --output table
az vm image list --publisher Canonical --offer 0001-com-ubuntu-server-jammy --all --output table
```
## VM Creation
### Linux VM with SSH Key
```bash
az vm create \
--resource-group mygroup \
--name myvm \
--resource-group compute-rg \
--name myapp-vm \
--image Ubuntu2204 \
--size Standard_D4s_v5 \
--admin-username azureuser \
--generate-ssh-keys \
--vnet-name myapp-vnet \
--subnet app-subnet \
--nsg "" \
--public-ip-address "" \
--os-disk-size-gb 64 \
--os-disk-caching ReadWrite \
--storage-sku Premium_LRS \
--zone 1 \
--assign-identity \
--tags environment=prod team=platform app=myapp
# SSH into the VM (if public IP assigned)
ssh azureuser@$(az vm show -g compute-rg -n myapp-vm -d --query publicIps -o tsv)
```
### Windows VM
```bash
az vm create \
--resource-group compute-rg \
--name myapp-win-vm \
--image Win2022Datacenter \
--size Standard_D4s_v5 \
--admin-username azureadmin \
--admin-password 'S3cur3P@ssw0rd!' \
--vnet-name myapp-vnet \
--subnet app-subnet \
--public-ip-address "" \
--os-disk-size-gb 128 \
--storage-sku Premium_LRS \
--zone 1
```
### VM with Cloud-Init
```bash
# cloud-init.yaml
# #cloud-config
# package_update: true
# packages:
# - nginx
# - docker.io
# runcmd:
# - systemctl enable nginx
# - systemctl start nginx
# - usermod -aG docker azureuser
az vm create \
--resource-group compute-rg \
--name web-vm \
--image Ubuntu2204 \
--size Standard_B2s \
--admin-username azureuser \
--generate-ssh-keys \
--nsg-rule SSH
--custom-data cloud-init.yaml \
--tags role=web
```
## Scale Sets
## VM Size Guide
| Family | Example Sizes | Use Case |
|--------|---------------|----------|
| B-series | Standard_B1s, Standard_B2s | Dev/test, low-traffic web servers |
| D-series | Standard_D4s_v5, Standard_D8s_v5 | General purpose, most production workloads |
| E-series | Standard_E4s_v5, Standard_E16s_v5 | Memory-intensive (databases, caching) |
| F-series | Standard_F4s_v2, Standard_F16s_v2 | CPU-intensive (batch processing, analytics) |
| L-series | Standard_L8s_v3, Standard_L32s_v3 | Storage-optimized (big data, SQL) |
| N-series | Standard_NC6s_v3, Standard_NC24ads_A100_v4 | GPU workloads (ML training, rendering) |
| M-series | Standard_M128s | SAP HANA, large in-memory workloads |
```bash
az vmss create \
--resource-group mygroup \
--name myvmss \
--image Ubuntu2204 \
--instance-count 2 \
--vm-sku Standard_B2s \
--upgrade-policy-mode automatic
# Find VM sizes with specific capabilities
az vm list-sizes --location eastus \
--query "[?numberOfCores >= \`4\` && memoryInMb >= \`16000\`]" \
--output table
# Check VM size availability in a zone
az vm list-skus --location eastus \
--size Standard_D4s_v5 \
--output table
```
## Best Practices
## Managed Disks
- Use managed disks
- Implement availability zones
- Use scale sets for auto-scaling
- Enable Azure Backup
- Use spot instances for cost savings
```bash
# Add a data disk to existing VM
az vm disk attach \
--resource-group compute-rg \
--vm-name myapp-vm \
--name myapp-data-disk \
--size-gb 256 \
--sku Premium_LRS \
--new \
--lun 0
# Create a standalone managed disk
az disk create \
--resource-group compute-rg \
--name shared-data-disk \
--size-gb 512 \
--sku Premium_LRS \
--zone 1
# Resize a disk (VM must be deallocated)
az vm deallocate --resource-group compute-rg --name myapp-vm
az disk update \
--resource-group compute-rg \
--name myapp-data-disk \
--size-gb 512
az vm start --resource-group compute-rg --name myapp-vm
# Snapshot a disk for backup
az snapshot create \
--resource-group compute-rg \
--name myapp-disk-snapshot \
--source myapp-data-disk
# Create disk from snapshot
az disk create \
--resource-group compute-rg \
--name myapp-disk-from-snap \
--source myapp-disk-snapshot \
--sku Premium_LRS
# List disks attached to a VM
az vm show \
--resource-group compute-rg \
--name myapp-vm \
--query "storageProfile.dataDisks" \
--output table
```
## Custom Images
```bash
# Generalize the VM (run inside the VM first)
# sudo waagent -deprovision+user -force
# Deallocate and generalize
az vm deallocate --resource-group compute-rg --name myapp-vm
az vm generalize --resource-group compute-rg --name myapp-vm
# Create image from VM
az image create \
--resource-group compute-rg \
--name myapp-golden-image \
--source myapp-vm \
--os-type Linux
# Create VM from custom image
az vm create \
--resource-group compute-rg \
--name myapp-from-image \
--image myapp-golden-image \
--size Standard_D4s_v5 \
--admin-username azureuser \
--generate-ssh-keys
# Use Azure Compute Gallery for shared images
az sig create \
--resource-group compute-rg \
--gallery-name myAppGallery
az sig image-definition create \
--resource-group compute-rg \
--gallery-name myAppGallery \
--gallery-image-definition myapp-image \
--publisher myorg \
--offer myapp \
--sku 1.0 \
--os-type Linux \
--os-state Generalized
az sig image-version create \
--resource-group compute-rg \
--gallery-name myAppGallery \
--gallery-image-definition myapp-image \
--gallery-image-version 1.0.0 \
--managed-image myapp-golden-image \
--target-regions eastus westus \
--replica-count 2
```
## Availability Sets and Zones
```bash
# Create availability set
az vm availability-set create \
--resource-group compute-rg \
--name myapp-avset \
--platform-fault-domain-count 3 \
--platform-update-domain-count 5
# Create VM in availability set
az vm create \
--resource-group compute-rg \
--name myapp-vm-1 \
--image Ubuntu2204 \
--size Standard_D4s_v5 \
--availability-set myapp-avset \
--admin-username azureuser \
--generate-ssh-keys
# Create VMs across availability zones
for zone in 1 2 3; do
az vm create \
--resource-group compute-rg \
--name "myapp-vm-zone${zone}" \
--image Ubuntu2204 \
--size Standard_D4s_v5 \
--zone "$zone" \
--admin-username azureuser \
--generate-ssh-keys \
--no-wait
done
```
## Virtual Machine Scale Sets
```bash
# Create VMSS with autoscaling
az vmss create \
--resource-group compute-rg \
--name myapp-vmss \
--image Ubuntu2204 \
--vm-sku Standard_D4s_v5 \
--instance-count 2 \
--admin-username azureuser \
--generate-ssh-keys \
--vnet-name myapp-vnet \
--subnet app-subnet \
--upgrade-policy-mode Rolling \
--health-probe "/" \
--load-balancer myapp-lb \
--zones 1 2 3 \
--custom-data cloud-init.yaml \
--tags environment=prod
# Configure autoscale rules
az monitor autoscale create \
--resource-group compute-rg \
--resource myapp-vmss \
--resource-type Microsoft.Compute/virtualMachineScaleSets \
--name myapp-autoscale \
--min-count 2 \
--max-count 20 \
--count 3
# Scale out when CPU > 70%
az monitor autoscale rule create \
--resource-group compute-rg \
--autoscale-name myapp-autoscale \
--condition "Percentage CPU > 70 avg 5m" \
--scale out 2
# Scale in when CPU < 30%
az monitor autoscale rule create \
--resource-group compute-rg \
--autoscale-name myapp-autoscale \
--condition "Percentage CPU < 30 avg 10m" \
--scale in 1
# Manual scale
az vmss scale \
--resource-group compute-rg \
--name myapp-vmss \
--new-capacity 5
# Update VMSS image
az vmss update \
--resource-group compute-rg \
--name myapp-vmss \
--set virtualMachineProfile.storageProfile.imageReference.version=latest
# Rolling upgrade of instances
az vmss rolling-upgrade start \
--resource-group compute-rg \
--name myapp-vmss
# List VMSS instances
az vmss list-instances \
--resource-group compute-rg \
--name myapp-vmss \
--output table
```
## VM Management Operations
```bash
# Start / Stop / Restart / Deallocate
az vm start --resource-group compute-rg --name myapp-vm
az vm stop --resource-group compute-rg --name myapp-vm
az vm restart --resource-group compute-rg --name myapp-vm
az vm deallocate --resource-group compute-rg --name myapp-vm
# Run command on a VM
az vm run-command invoke \
--resource-group compute-rg \
--name myapp-vm \
--command-id RunShellScript \
--scripts "df -h && free -m && uptime"
# Enable boot diagnostics
az vm boot-diagnostics enable \
--resource-group compute-rg \
--name myapp-vm
# Get boot diagnostics log
az vm boot-diagnostics get-boot-log \
--resource-group compute-rg \
--name myapp-vm
# Enable Azure Backup
az backup protection enable-for-vm \
--resource-group compute-rg \
--vault-name myapp-vault \
--vm myapp-vm \
--policy-name DefaultPolicy
# Resize a VM
az vm resize \
--resource-group compute-rg \
--name myapp-vm \
--size Standard_D8s_v5
```
## Terraform Configuration
```hcl
resource "azurerm_linux_virtual_machine" "main" {
name = "myapp-vm"
resource_group_name = azurerm_resource_group.main.name
location = azurerm_resource_group.main.location
size = "Standard_D4s_v5"
admin_username = "azureuser"
zone = "1"
network_interface_ids = [azurerm_network_interface.main.id]
admin_ssh_key {
username = "azureuser"
public_key = file("~/.ssh/id_rsa.pub")
}
os_disk {
caching = "ReadWrite"
storage_account_type = "Premium_LRS"
disk_size_gb = 64
}
source_image_reference {
publisher = "Canonical"
offer = "0001-com-ubuntu-server-jammy"
sku = "22_04-lts-gen2"
version = "latest"
}
identity {
type = "SystemAssigned"
}
tags = var.tags
}
resource "azurerm_managed_disk" "data" {
name = "myapp-data-disk"
resource_group_name = azurerm_resource_group.main.name
location = azurerm_resource_group.main.location
storage_account_type = "Premium_LRS"
create_option = "Empty"
disk_size_gb = 256
zone = "1"
tags = var.tags
}
resource "azurerm_virtual_machine_data_disk_attachment" "data" {
managed_disk_id = azurerm_managed_disk.data.id
virtual_machine_id = azurerm_linux_virtual_machine.main.id
lun = 0
caching = "ReadOnly"
}
resource "azurerm_linux_virtual_machine_scale_set" "main" {
name = "myapp-vmss"
resource_group_name = azurerm_resource_group.main.name
location = azurerm_resource_group.main.location
sku = "Standard_D4s_v5"
instances = 3
admin_username = "azureuser"
zones = [1, 2, 3]
admin_ssh_key {
username = "azureuser"
public_key = file("~/.ssh/id_rsa.pub")
}
source_image_reference {
publisher = "Canonical"
offer = "0001-com-ubuntu-server-jammy"
sku = "22_04-lts-gen2"
version = "latest"
}
os_disk {
caching = "ReadWrite"
storage_account_type = "Premium_LRS"
}
network_interface {
name = "vmss-nic"
primary = true
ip_configuration {
name = "internal"
primary = true
subnet_id = azurerm_subnet.app.id
}
}
automatic_os_upgrade_policy {
disable_automatic_rollback = false
enable_automatic_os_upgrade = true
}
rolling_upgrade_policy {
max_batch_instance_percent = 20
max_unhealthy_instance_percent = 20
max_unhealthy_upgraded_instance_percent = 5
pause_time_between_batches = "PT0S"
}
tags = var.tags
}
```
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| VM fails to start | Quota exceeded in region | Check quota with `az vm list-usage --location eastus`; request increase |
| SSH connection refused | NSG blocking port 22 or VM not running | Check NSG rules and VM power state; use Azure Bastion for private VMs |
| VM disk full | OS disk too small or logs not rotated | Resize disk after deallocating; configure log rotation |
| VM performance is slow | Wrong VM size or disk throttling | Check metrics with `az monitor metrics list`; upgrade size or disk tier |
| Scale set not scaling out | Autoscale rule threshold not met | Review autoscale settings; verify metric thresholds match workload |
| Custom image VM boot fails | Image not properly generalized | Re-run `waagent -deprovision` before capturing; check boot diagnostics |
| VMSS rolling upgrade stuck | Health probe failing on new instances | Fix application health endpoint; check `az vmss rolling-upgrade get-latest` |
| Spot VM evicted unexpectedly | Azure reclaimed capacity | Use eviction policy `Deallocate` and set up eviction notifications |
## Related Skills
- `azure-networking` -- VNet and NSG configuration for VM connectivity.
- `azure-aks` -- Container alternative when VMs are not required.
- `arm-templates` -- Bicep-based VM deployment templates.
- `terraform-azure` -- Terraform-based VM and VMSS provisioning.
infrastructure/cloud-azure/terraform-azure/SKILL.md
+573 -21
View File
@@ -9,51 +9,603 @@ metadata:
# Terraform Azure
Provision Azure infrastructure with Terraform.
Provision and manage Azure infrastructure with Terraform using the AzureRM provider. Covers provider configuration, remote state, resource groups, VNets, AKS, Key Vault, complete .tf file examples, and production workflows.
## When to Use
- You need multi-cloud or cloud-agnostic Infrastructure as Code.
- Your team standardizes on Terraform across AWS, Azure, and GCP.
- You need plan/apply workflows with change preview before deployment.
- You want modular, reusable infrastructure components.
- You need state locking and drift detection for production infrastructure.
## Prerequisites
```bash
# Install Terraform
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform
# Verify installation
terraform version
# Install Azure CLI and login
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
az login
az account set --subscription "my-subscription-id"
# Create storage account for remote state
az group create --name tfstate-rg --location eastus
az storage account create \
--name tfstate$(openssl rand -hex 4) \
--resource-group tfstate-rg \
--sku Standard_LRS \
--encryption-services blob
az storage container create \
--name tfstate \
--account-name tfstateXXXXXXXX
```
## Provider Configuration
### providers.tf
```hcl
terraform {
required_version = ">= 1.5.0"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.0"
version = "~> 3.80"
}
azuread = {
source = "hashicorp/azuread"
version = "~> 2.47"
}
random = {
source = "hashicorp/random"
version = "~> 3.6"
}
}
backend "azurerm" {
resource_group_name = "tfstate"
storage_account_name = "tfstate12345"
resource_group_name = "tfstate-rg"
storage_account_name = "tfstate12345abc"
container_name = "tfstate"
key = "prod.terraform.tfstate"
}
}
provider "azurerm" {
features {}
features {
key_vault {
purge_soft_delete_on_destroy = false
recover_soft_deleted_key_vaults = true
}
resource_group {
prevent_deletion_if_contains_resources = true
}
}
# Optional: use a specific subscription
# subscription_id = var.subscription_id
}
provider "azuread" {}
```
### variables.tf
```hcl
variable "environment" {
description = "Environment name (dev, staging, prod)"
type = string
validation {
condition = contains(["dev", "staging", "prod"], var.environment)
error_message = "Environment must be dev, staging, or prod."
}
}
variable "location" {
description = "Azure region for all resources"
type = string
default = "eastus"
}
variable "project_name" {
description = "Project name used in resource naming"
type = string
default = "myapp"
}
variable "tags" {
description = "Tags applied to all resources"
type = map(string)
default = {}
}
variable "sql_admin_password" {
description = "SQL Server admin password"
type = string
sensitive = true
}
variable "aks_admin_group_id" {
description = "Azure AD group ID for AKS admin access"
type = string
}
locals {
name_prefix = "${var.project_name}-${var.environment}"
common_tags = merge(var.tags, {
environment = var.environment
project = var.project_name
managed_by = "terraform"
})
}
```
## Example Resources
### terraform.tfvars (per environment)
```hcl
# terraform.prod.tfvars
environment = "prod"
location = "eastus"
project_name = "myapp"
aks_admin_group_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
tags = {
cost_center = "engineering"
owner = "platform-team"
}
```
## Resource Group
### resource-group.tf
```hcl
resource "azurerm_resource_group" "main" {
name = "myapp-rg"
location = "East US"
}
resource "azurerm_virtual_network" "main" {
name = "myapp-vnet"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
name = "${local.name_prefix}-rg"
location = var.location
tags = local.common_tags
}
```
## Best Practices
## Virtual Network
- Use remote state in Azure Storage
- Implement resource naming conventions
- Use data sources for existing resources
- Tag all resources
- Use modules for reusability
### network.tf
```hcl
resource "azurerm_virtual_network" "main" {
name = "${local.name_prefix}-vnet"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
address_space = ["10.0.0.0/16"]
tags = local.common_tags
}
resource "azurerm_subnet" "aks" {
name = "aks-subnet"
resource_group_name = azurerm_resource_group.main.name
virtual_network_name = azurerm_virtual_network.main.name
address_prefixes = ["10.0.1.0/22"]
}
resource "azurerm_subnet" "app" {
name = "app-subnet"
resource_group_name = azurerm_resource_group.main.name
virtual_network_name = azurerm_virtual_network.main.name
address_prefixes = ["10.0.8.0/24"]
}
resource "azurerm_subnet" "data" {
name = "data-subnet"
resource_group_name = azurerm_resource_group.main.name
virtual_network_name = azurerm_virtual_network.main.name
address_prefixes = ["10.0.9.0/24"]
private_endpoint_network_policies_enabled = true
}
resource "azurerm_network_security_group" "app" {
name = "${local.name_prefix}-app-nsg"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
security_rule {
name = "AllowHTTPS"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "443"
source_address_prefix = "*"
destination_address_prefix = "*"
}
tags = local.common_tags
}
resource "azurerm_subnet_network_security_group_association" "app" {
subnet_id = azurerm_subnet.app.id
network_security_group_id = azurerm_network_security_group.app.id
}
```
## AKS Cluster
### aks.tf
```hcl
resource "azurerm_log_analytics_workspace" "aks" {
name = "${local.name_prefix}-law"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
sku = "PerGB2018"
retention_in_days = 30
tags = local.common_tags
}
resource "azurerm_kubernetes_cluster" "main" {
name = "${local.name_prefix}-aks"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
dns_prefix = "${var.project_name}-${var.environment}"
kubernetes_version = "1.28"
default_node_pool {
name = "system"
vm_size = "Standard_D4s_v5"
enable_auto_scaling = true
min_count = 2
max_count = 5
zones = [1, 2, 3]
vnet_subnet_id = azurerm_subnet.aks.id
os_disk_size_gb = 128
os_disk_type = "Managed"
max_pods = 50
node_labels = {
role = "system"
}
}
identity {
type = "SystemAssigned"
}
network_profile {
network_plugin = "azure"
network_policy = "calico"
service_cidr = "10.1.0.0/16"
dns_service_ip = "10.1.0.10"
load_balancer_sku = "standard"
}
azure_active_directory_role_based_access_control {
managed = true
azure_rbac_enabled = true
admin_group_object_ids = [var.aks_admin_group_id]
}
oms_agent {
log_analytics_workspace_id = azurerm_log_analytics_workspace.aks.id
}
key_vault_secrets_provider {
secret_rotation_enabled = true
secret_rotation_interval = "2m"
}
tags = local.common_tags
}
resource "azurerm_kubernetes_cluster_node_pool" "app" {
name = "app"
kubernetes_cluster_id = azurerm_kubernetes_cluster.main.id
vm_size = "Standard_D8s_v5"
enable_auto_scaling = true
min_count = 2
max_count = 20
zones = [1, 2, 3]
vnet_subnet_id = azurerm_subnet.aks.id
max_pods = 50
node_labels = {
workload = "app"
}
node_taints = [
"dedicated=app:NoSchedule"
]
tags = local.common_tags
}
```
## Key Vault
### keyvault.tf
```hcl
data "azurerm_client_config" "current" {}
resource "azurerm_key_vault" "main" {
name = "${var.project_name}-${var.environment}-kv"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
tenant_id = data.azurerm_client_config.current.tenant_id
sku_name = "standard"
soft_delete_retention_days = 90
purge_protection_enabled = true
enabled_for_disk_encryption = true
network_acls {
default_action = "Deny"
bypass = "AzureServices"
ip_rules = var.allowed_ip_ranges
virtual_network_subnet_ids = [
azurerm_subnet.app.id,
azurerm_subnet.aks.id,
]
}
tags = local.common_tags
}
resource "azurerm_key_vault_access_policy" "terraform" {
key_vault_id = azurerm_key_vault.main.id
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = data.azurerm_client_config.current.object_id
secret_permissions = [
"Get", "List", "Set", "Delete", "Purge", "Recover"
]
key_permissions = [
"Get", "List", "Create", "Delete", "Purge", "Recover",
"WrapKey", "UnwrapKey"
]
}
resource "azurerm_key_vault_access_policy" "aks" {
key_vault_id = azurerm_key_vault.main.id
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = azurerm_kubernetes_cluster.main.key_vault_secrets_provider[0].secret_identity[0].object_id
secret_permissions = ["Get", "List"]
}
resource "azurerm_key_vault_secret" "sql_password" {
name = "sql-admin-password"
value = var.sql_admin_password
key_vault_id = azurerm_key_vault.main.id
depends_on = [azurerm_key_vault_access_policy.terraform]
}
```
## SQL Database
### database.tf
```hcl
resource "azurerm_mssql_server" "main" {
name = "${local.name_prefix}-sql"
resource_group_name = azurerm_resource_group.main.name
location = azurerm_resource_group.main.location
version = "12.0"
administrator_login = "sqladmin"
administrator_login_password = var.sql_admin_password
minimum_tls_version = "1.2"
azuread_administrator {
login_username = "SQL Admins"
object_id = var.aks_admin_group_id
}
tags = local.common_tags
}
resource "azurerm_mssql_database" "main" {
name = "${var.project_name}-db"
server_id = azurerm_mssql_server.main.id
collation = "SQL_Latin1_General_CP1_CI_AS"
sku_name = var.environment == "prod" ? "BC_Gen5_4" : "GP_S_Gen5_2"
max_size_gb = var.environment == "prod" ? 256 : 32
zone_redundant = var.environment == "prod"
short_term_retention_policy {
retention_days = var.environment == "prod" ? 14 : 7
}
tags = local.common_tags
}
resource "azurerm_private_endpoint" "sql" {
name = "${local.name_prefix}-sql-pe"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
subnet_id = azurerm_subnet.data.id
private_service_connection {
name = "sql-connection"
private_connection_resource_id = azurerm_mssql_server.main.id
subresource_names = ["sqlServer"]
is_manual_connection = false
}
tags = local.common_tags
}
```
## Outputs
### outputs.tf
```hcl
output "resource_group_name" {
value = azurerm_resource_group.main.name
}
output "aks_cluster_name" {
value = azurerm_kubernetes_cluster.main.name
}
output "aks_kube_config" {
value = azurerm_kubernetes_cluster.main.kube_config_raw
sensitive = true
}
output "key_vault_uri" {
value = azurerm_key_vault.main.vault_uri
}
output "sql_server_fqdn" {
value = azurerm_mssql_server.main.fully_qualified_domain_name
}
output "vnet_id" {
value = azurerm_virtual_network.main.id
}
```
## Terraform Workflow Commands
```bash
# Initialize (download providers, configure backend)
terraform init
# Validate configuration syntax
terraform validate
# Format all .tf files
terraform fmt -recursive
# Plan changes for a specific environment
terraform plan \
-var-file="terraform.prod.tfvars" \
-var="sql_admin_password=$(az keyvault secret show --vault-name ops-vault --name sql-pass --query value -o tsv)" \
-out=tfplan
# Apply the saved plan
terraform apply tfplan
# Apply with auto-approve (CI/CD pipelines only)
terraform apply \
-var-file="terraform.prod.tfvars" \
-auto-approve
# Destroy infrastructure (careful!)
terraform plan -destroy -var-file="terraform.prod.tfvars" -out=destroyplan
terraform apply destroyplan
# Import existing resources into state
terraform import azurerm_resource_group.main /subscriptions/{sub}/resourceGroups/myapp-prod-rg
# Show current state
terraform state list
terraform state show azurerm_kubernetes_cluster.main
# Move resources in state (renaming)
terraform state mv azurerm_resource_group.old azurerm_resource_group.new
# Refresh state from real infrastructure
terraform refresh -var-file="terraform.prod.tfvars"
# Unlock stuck state
terraform force-unlock LOCK_ID
# Use workspaces for environment isolation
terraform workspace new prod
terraform workspace select prod
terraform workspace list
```
## Module Structure
```
project/
modules/
networking/
main.tf
variables.tf
outputs.tf
aks/
main.tf
variables.tf
outputs.tf
database/
main.tf
variables.tf
outputs.tf
environments/
dev/
main.tf
terraform.tfvars
backend.tf
prod/
main.tf
terraform.tfvars
backend.tf
```
### Using Modules
```hcl
# environments/prod/main.tf
module "networking" {
source = "../../modules/networking"
environment = var.environment
location = var.location
project_name = var.project_name
address_space = ["10.0.0.0/16"]
}
module "aks" {
source = "../../modules/aks"
environment = var.environment
location = var.location
project_name = var.project_name
resource_group_name = module.networking.resource_group_name
subnet_id = module.networking.aks_subnet_id
admin_group_id = var.aks_admin_group_id
}
module "database" {
source = "../../modules/database"
environment = var.environment
location = var.location
project_name = var.project_name
resource_group_name = module.networking.resource_group_name
subnet_id = module.networking.data_subnet_id
admin_password = var.sql_admin_password
}
```
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| `Error acquiring state lock` | Previous run crashed or concurrent access | Run `terraform force-unlock LOCK_ID` after confirming no other run is active |
| `Provider version constraint error` | Version conflict in required_providers | Run `terraform init -upgrade` to fetch compatible versions |
| `Resource already exists` | Resource created outside Terraform | Import with `terraform import` to bring it under management |
| `Cycle detected` in plan | Circular dependency between resources | Restructure references or use `depends_on` carefully |
| State file corruption | Concurrent writes or manual edits | Restore from state backup in the storage account versioning |
| `AuthorizationFailed` during apply | Service principal lacks RBAC permissions | Assign Contributor role on subscription or resource group |
| Plan shows unexpected changes | Drift from manual portal changes | Run `terraform refresh` then `terraform plan` to reconcile |
| Module source not found | Incorrect relative path or registry reference | Verify path in `source` attribute; run `terraform init` again |
## Related Skills
- `arm-templates` -- Azure-native IaC alternative with Bicep.
- `azure-aks` -- AKS cluster details and kubectl operations.
- `azure-networking` -- VNet and NSG design referenced in Terraform configs.
- `azure-sql` -- Database provisioning and security configurations.
- `azure-vms` -- VM sizing and scale set configurations.