Terraform Iac Mastery

Category: DevOps & Infrastructure | Read: 27 min | v1.0.0

Terraform & Infrastructure as Code Mastery

Table of Contents

  • Overview & Tool Landscape
  • HCL Syntax & Core Concepts
  • Providers — AWS, GCP, Azure & Multi-Cloud
  • Resources
  • Data Sources
  • Variables — Input, Local, Output
  • State Management
  • Workspaces for Environments
  • Modules — Structure, Versioning & Registry
  • Provisioners — remote-exec & local-exec
  • Dynamic Blocks
  • for_each & count
  • Conditionals
  • Terraform Plan / Apply / Destroy Workflow
  • Import Existing Resources
  • State Manipulation — mv, rm, import
  • Drift Detection
  • Sentinel & OPA Policy
  • Workspaces Pattern
  • Best Practices
  • Common Pitfalls
  • Quick Reference — Commands

    • Overview & Tool Landscape

    Infrastructure as Code (IaC) manages infrastructure through machine-readable definition files rather than manual processes. The major IaC tools:

    | Tool | Language | State | Multi-Cloud | Key Strength |
    |------|----------|-------|-------------|--------------|
    | Terraform | HCL | Yes | Yes | Largest ecosystem, provider registry |
    | Pulumi | Python/Go/TS/Java | Yes | Yes | Real programming languages |
    | CloudFormation | YAML/JSON | Yes | AWS only | Deep AWS integration |
    | CDK for Terraform (CDKTF) | TS/Python/Java/Go | Yes | Yes | Terraform with programming languages |
    | Ansible | YAML | No | Yes | Config management, agentless |

    This skill focuses primarily on Terraform (the industry standard) with cross-references to Pulumi and CloudFormation where relevant.

    HCL Syntax & Core Concepts

    HCL (HashiCorp Configuration Language) is Terraform's declarative language.

    Basic Structure

    # Block type, labels, body
block_type "label1" "label2" {
  argument = value
}

    Data Types

    # Primitives
string_var  = "hello world"
number_var  = 42
bool_var    = true

    Collections
    
list_var    = ["a", "b", "c"]
map_var    = { key1 = "val1", key2 = "val2" }
set_var   = toset(["a", "b", "c"])  # unique elements

    Complex
    
object_var = {
  name = "example"
  port = 443
}
tuple_var  = ["hello", 42, true]  # mixed types, fixed order

    Expressions & Operators

    # String interpolation
"Hello, ${var.name}!"

    Conditional expression
    
var.enabled ? "yes" : "no"

    For expressions
    
[for s in var.list : upper(s)]
{for s in var.list : s => upper(s)}
[for k, v in var.map : "${k}=${v}"]

    Splat expressions
    
instance.*.id           # legacy splat
instance[*].id          # full splat (recommended)

    Dynamic type conversion
    
tonumber("42")       # -> 42
tostring(42)         # -> "42"
tolist(["a","b"])
toset(["a","b"])
tomap({a="1"})

    Comments

    # Single-line comment (preferred)
// Single-line comment (also valid)
/* Multi-line
   comment */

    Locals Block

    locals {
  name_prefix = "${var.project}-${var.environment}"
  common_tags = {
    Environment = var.environment
    Project     = var.project
    ManagedBy   = "terraform"
  }
}

    Providers — AWS, GCP, Azure & Multi-Cloud

    Providers are plugins that Terraform uses to manage resources across different cloud platforms and services.

    Provider Configuration

    terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
    google = {
      source  = "hashicorp/google"
      version = "~> 5.0"
    }
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0"
    }
  }
  required_version = ">= 1.5.0"
}

    AWS Provider

    provider "aws" {
  region  = var.aws_region
  profile = "production"

    # Multiple regions
      alias = "us_east_1"
    }
    

    provider "aws" {
      alias  = "us_east_1"
      region = "us-east-1"
    }

    GCP Provider

    provider "google" {
  project = var.gcp_project
  region  = var.gcp_region
  zone    = var.gcp_zone

    credentials = file(var.credentials_path)
    }

    Azure Provider

    provider "azurerm" {
  features {}

    subscription_id = var.subscription_id
      tenant_id       = var.tenant_id
      client_id       = var.client_id
      client_secret   = var.client_secret
    }

    Multi-Cloud Patterns

    # Use different providers for different resources
resource "aws_s3_bucket" "data" {
  provider = aws.us_east_1
  bucket   = "${var.project}-data"
}

    resource "google_storage_bucket" "data" {
      provider = google
      name     = "${var.project}-data"
      location = var.gcp_region
    }

    Provider Dependency Lock

    # Initialize and create .terraform.lock.hcl
terraform init

    Update a single provider
    
terraform init -upgrade=aws

    Pin provider versions
    
terraform providers lock \
  -platform=linux_amd64 \
  -platform=darwin_arm64

    Resources

    Resources are the core building blocks — each declares a piece of infrastructure.

    resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = var.instance_type

    tags = merge(local.common_tags, {
        Name = "${local.name_prefix}-web"
      })
    }

    Resource Behavior

  • Create: Resource doesn't exist → Terraform creates it
  • Update in-place: Attribute change supports in-place update → modified without replacement
  • Force replacement: Attribute change requires replacement → destroy + recreate
  • Destroy: Resource removed from config → Terraform destroys it

    • Lifecycle Meta-Arguments

    resource "aws_rds_cluster" "main" {
  # ...

    lifecycle {
        create_before_destroy = true   # Replace: create new first
        prevent_destroy       = true   # Protect from accidental deletion
        ignore_changes = [             # Ignore specific attribute drift
          instance_class,
          tags["LastModified"],
        ]
        replace_triggered_by = [       # Replace when referenced resource changes
          aws_db_parameter_group.main
        ]
      }
    }

    Depends On

    resource "aws_instance" "web" {
  # ...

    depends_on = [
        aws_iam_role_policy.example,
        aws_internet_gateway.gw,
      ]
    }

    Timeouts

    resource "aws_db_instance" "main" {
  # ...

    timeouts {
        create = "60m"
        update = "40m"
        delete = "20m"
      }
    }

    Data Sources

    Data sources read existing infrastructure or compute values.

    # Look up latest AMI
data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"]  # Canonical

    filter {
        name   = "name"
        values = ["ubuntu/images/hvm-ssd/ubuntu--${var.ubuntu_version}-amd64-server-"]
      }
    

    filter {
        name   = "virtualization-type"
        values = ["hvm"]
      }
    }
    

    Look up VPC
    
data "aws_vpc" "main" {
  filter {
    name   = "tag:Name"
    values = ["production-vpc"]
  }
}

    Look up availability zones
    
data "aws_availability_zones" "available" {
  state = "available"
}

    Use data source
    
resource "aws_subnet" "main" {
  vpc_id            = data.aws_vpc.main.id
  availability_zone  = data.aws_availability_zones.available.names[0]
}

    Terraform Data Source (built-in)

    data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "terraform-state-prod"
    key    = "network/terraform.tfstate"
    region = "us-east-1"
  }
}

    Reference outputs from another state
    
resource "aws_instance" "app" {
  subnet_id = data.terraform_remote_state.network.outputs.subnet_ids[0]
}

    Variables — Input, Local, Output

    Input Variables

    # variables.tf
variable "project" {
  description = "Project name used for naming resources"
  type        = string
  default     = "myapp"
}

    variable "environment" {
      description = "Deployment environment"
      type        = string
    

    validation {
        condition     = contains(["dev", "staging", "prod"], var.environment)
        error_message = "Environment must be dev, staging, or prod."
      }
    }
    

    variable "instance_type" {
      description = "EC2 instance type"
      type        = string
      default     = "t3.micro"
    }
    

    variable "common_tags" {
      description = "Common tags applied to all resources"
      type        = map(string)
      default     = {}
    }
    

    variable "subnet_cidrs" {
      description = "CIDR blocks for subnets"
      type        = list(string)
      default     = ["10.0.1.0/24", "10.0.2.0/24"]
    }
    

    variable "sensitive_data" {
      description = "Secret value — will not appear in logs"
      type        = string
      sensitive   = true
    }
    

    variable "nullable_var" {
      type     = string
      nullable = true  # Allows null as a value
    }

    Variable Precedence (highest to lowest)

  • -var flag: terraform apply -var="env=prod"
  • -var-file: terraform apply -var-file=prod.tfvars
  • TF_VAR_ environment variables: export TF_VAR_env=prod
  • terraform.tfvars (auto-loaded)
  • *.auto.tfvars (auto-loaded)
  • default in variable block

    • tfvars Files

    # dev.tfvars
environment   = "dev"
instance_type = "t3.micro"
subnet_cidrs  = ["10.0.1.0/24", "10.0.2.0/24"]

    Local Values

    locals {
  name_prefix = "${var.project}-${var.environment}"

    subnet_ids = aws_subnet.main[*].id
    

    # Complex computation
      natural_sort = sort(var.names)
    

    # Grouping related values
      default_tags = merge(var.common_tags, {
        Environment = var.environment
        ManagedBy   = "terraform"
        Timestamp   = timestamp()
      })
    }

    Output Values

    # outputs.tf
output "vpc_id" {
  description = "The VPC ID"
  value       = aws_vpc.main.id
}

    output "instance_ips" {
      description = "Public IPs of instances"
      value       = aws_instance.web[*].public_ip
    }
    

    output "connection_string" {
      description = "Database connection string"
      value       = "postgres://${var.db_user}:${var.db_pass}@${aws_db_instance.main.endpoint}"
      sensitive   = true  # Marked as sensitive
    }
    

    output "role_arn" {
      value = aws_iam_role.main.arn
    

    # Prevent exposing in root module outputs
      # (useful for intermediate modules)
    }

    State Management

    State maps real-world resources to your configuration, tracks metadata, and improves performance.

    Remote Backends

    #### S3 Backend (AWS)

    terraform {
  backend "s3" {
    bucket         = "terraform-state-prod"
    key            = "infra/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"
    kms_key_id     = "arn:aws:kms:us-east-1:123456789012:key/xxx"
  }
}

    #### GCS Backend (GCP)

    terraform {
  backend "gcs" {
    bucket = "terraform-state-prod"
    prefix = "infra/terraform"
  }
}

    #### Azure Blob Backend

    terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-rg"
    storage_account_name = "tfstateprod"
    container_name       = "tfstate"
    key                  = "infra.terraform.tfstate"
  }
}

    #### Consul Backend

    terraform {
  backend "consul" {
    path = "terraform/infra"
  }
}

    State Locking

    State locking prevents concurrent operations. Supported backends:

  • S3 + DynamoDB: Most common AWS pattern

  • GCS: Built-in locking

  • Azure Blob: Built-in lease-based locking

  • Consul: Built-in locking via sessions

    • State Commands

    # View current state
terraform state list

    Show resource details
    
terraform state show aws_instance.web

    Pull state for inspection
    
terraform state pull > state.json

    Push state (dangerous — use with caution)
    
terraform state push state.json

    Remove resource from state (does not destroy real resource)
    
terraform state rm aws_instance.web

    Workspaces for Environments

    Workspaces allow multiple named instances of the same configuration with separate state.

    # List workspaces
terraform workspace list

    Create a workspace
    
terraform workspace new staging

    Switch workspace
    
terraform workspace select prod

    Show current workspace
    
terraform workspace show

    Delete workspace
    
terraform workspace delete staging

    Using Workspace Values

    # Workspace-based configuration
variable "environment_config" {
  type = map(object({
    instance_type = string
    min_size      = number
    max_size      = number
  }))
  default = {
    dev     = { instance_type = "t3.micro",    min_size = 1, max_size = 2 }
    staging = { instance_type = "t3.medium",   min_size = 2, max_size = 4 }
    prod    = { instance_type = "t3.xlarge",   min_size = 4, max_size = 10 }
  }
}

    locals {
      env_config = var.environment_config[terraform.workspace]
    }
    

    resource "aws_instance" "app" {
      instance_type = local.env_config.instance_type
      # ...
    }

    Workspace Naming in Resources

    resource "aws_s3_bucket" "app" {
  bucket = "${var.project}-${terraform.workspace}"
}

    > Warning: Workspaces share the same code — use directory-based separation for fundamentally different environments (e.g., different clouds).

    Modules — Structure, Versioning & Registry

    Module Structure

    modules/
└── vpc/
    ├── main.tf          # Primary resource definitions
    ├── variables.tf     # Input variable declarations
    ├── outputs.tf       # Output value declarations
    ├── versions.tf      # Provider & Terraform version constraints
    ├── locals.tf        # Local values (optional)
    ├── data.tf          # Data sources (optional)
    └── README.md        # Module documentation

    Using Modules

    module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

    name = "${var.project}-vpc"
      cidr = var.vpc_cidr
    

    azs             = var.availability_zones
      private_subnets = var.private_subnet_cidrs
      public_subnets  = var.public_subnet_cidrs
    

    enable_nat_gateway = true
      single_nat_gateway = var.environment != "prod"
    

    tags = local.common_tags
    }

    Module Sources

    # Terraform Registry
source  = "terraform-aws-modules/vpc/aws"

    Local path
    
source  = "./modules/vpc"

    Git repository
    
source  = "git::https://github.com/org/terraform-modules.git//vpc"

    Git branch/tag
    
source  = "git::https://github.com/org/terraform-modules.git//vpc?ref=v2.0.0"

    S3 bucket
    
source  = "s3::https://s3.amazonaws.com/bucket/modules/vpc.zip"

    GitHub
    
source  = "github.com/org/terraform-modules//vpc"

    Module Versioning

    module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.0.0"     # Exact version
}

    module "vpc" {
      source  = "terraform-aws-modules/vpc/aws"
      version = "~> 5.0"   # Any 5.x.x (>= 5.0.0, < 6.0.0)
    }
    

    module "vpc" {
      source  = "terraform-aws-modules/vpc/aws"
      version = ">= 5.0"   # Any version >= 5.0
    }

    Creating a Reusable Module

    # modules/vpc/variables.tf
variable "cidr" {
  description = "CIDR block for the VPC"
  type        = string
  default     = "10.0.0.0/16"
}

    variable "enable_nat_gateway" {
      description = "Enable NAT Gateway"
      type        = bool
      default     = true
    }
    

    modules/vpc/main.tf
    
resource "aws_vpc" "main" {
  cidr_block = var.cidr
  tags = merge(var.tags, {
    Name = var.name
  })
}

    modules/vpc/outputs.tf
    
output "vpc_id" {
  value = aws_vpc.main.id
}

    output "vpc_cidr" {
      value = aws_vpc.main.cidr_block
    }

    Module Best Practices

  • Single responsibility: Each module manages one logical component
  • Expose outputs: Provide all necessary outputs for downstream consumers
  • Use defaults: Sensible defaults reduce required inputs
  • Version pin: Always pin module versions in production
  • Validate inputs: Use validation blocks on variables
  • Document: README with inputs, outputs, and examples

    • Provisioners — remote-exec & local-exec

    > Note: Provisioners are a last resort. Prefer cloud-init, user data, or configuration management.

    remote-exec

    resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"

    provisioner "remote-exec" {
        inline = [
          "sudo apt-get update",
          "sudo apt-get install -y nginx",
          "sudo systemctl enable nginx",
        ]
      }
    

    connection {
        type        = "ssh"
        user        = "ubuntu"
        private_key = file("~/.ssh/id_rsa")
        host        = self.public_ip
      }
    }

    local-exec

    resource "aws_instance" "web" {
  # ...

    provisioner "local-exec" {
        command = "echo ${self.public_ip} > inventory.txt"
      }
    

    # With interpreter specification
      provisioner "local-exec" {
        command     = "python3 update_inventory.py ${self.public_ip}"
        interpreter = ["/usr/bin/python3", "-c"]
      }
    

    # When = destroy (runs on destroy)
      provisioner "local-exec" {
        when    = destroy
        command = "python3 cleanup.py ${self.id}"
      }
    }

    file Provisioner

    resource "aws_instance" "web" {
  # ...

    provisioner "file" {
        source      = "configs/app.conf"
        destination = "/etc/app/app.conf"
    

    connection {
          type        = "ssh"
          user        = "ubuntu"
          private_key = file("~/.ssh/id_rsa")
          host        = self.public_ip
        }
      }
    }

    Failure Handling

    resource "aws_instance" "web" {
  # ...

    provisioner "remote-exec" {
        on_failure = continue  # or fail (default)
        inline     = ["sudo apt-get update"]
      }
    }

    Dynamic Blocks

    Dynamic blocks generate nested blocks programmatically.

    resource "aws_security_group" "main" {
  name = "${var.project}-sg"

    # Dynamic ingress rules
      dynamic "ingress" {
        for_each = var.ingress_rules
        content {
          from_port   = ingress.value.port
          to_port     = ingress.value.port
          protocol    = ingress.value.protocol
          cidr_blocks = ingress.value.cidr_blocks
          description = ingress.value.description
        }
      }
    

    # Dynamic egress rules
      dynamic "egress" {
        for_each = var.egress_rules
        content {
          from_port   = egress.value.from_port
          to_port     = egress.value.to_port
          protocol    = egress.value.protocol
          cidr_blocks = egress.value.cidr_blocks
        }
      }
    }
    

    variable "ingress_rules" {
      type = list(object({
        port        = number
        protocol    = string
        cidr_blocks = list(string)
        description = string
      }))
      default = [
        { port = 80, protocol = "tcp", cidr_blocks = ["0.0.0.0/0"], description = "HTTP" },
        { port = 443, protocol = "tcp", cidr_blocks = ["0.0.0.0/0"], description = "HTTPS" },
        { port = 22, protocol = "tcp", cidr_blocks = ["10.0.0.0/8"], description = "SSH" },
      ]
    }

    Dynamic Block with Iterator

    dynamic "label" {
  for_each = var.labels
  iterator = item     # Custom iterator name

    content {
        key   = item.key
        value = item.value
      }
    }

    for_each & count

    count — Index-Based Replication

    variable "subnet_cidrs" {
  default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
}

    resource "aws_subnet" "main" {
      count             = length(var.subnet_cidrs)
      vpc_id            = aws_vpc.main.id
      availability_zone  = data.aws_availability_zones.available.names[count.index]
      cidr_block        = var.subnet_cidrs[count.index]
    

    tags = {
        Name = "${var.project}-subnet-${count.index}"
      }
    }

    for_each — Key-Based Replication (Preferred)

    variable "subnets" {
  type = map(object({
    cidr = string
    az   = string
  }))
  default = {
    subnet_a = { cidr = "10.0.1.0/24", az = "us-east-1a" }
    subnet_b = { cidr = "10.0.2.0/24", az = "us-east-1b" }
    subnet_c = { cidr = "10.0.3.0/24", az = "us-east-1c" }
  }
}

    resource "aws_subnet" "main" {
      for_each = var.subnets
    

    vpc_id            = aws_vpc.main.id
      availability_zone  = each.value.az
      cidr_block        = each.value.cidr
    

    tags = {
        Name = "${var.project}-${each.key}"
      }
    }

    for_each with toset

    variable "environments" {
  type    = set(string)
  default = ["dev", "staging", "prod"]
}

    resource "aws_s3_bucket" "logs" {
      for_each = toset(var.environments)
      bucket   = "${var.project}-logs-${each.value}"
    }

    Key Differences

    | Feature | count | for_each |
    |---------|---------|------------|
    | Key type | Integer index | String key |
    | Removing middle item | Shifts all indices ⚠️ | No effect on others ✅ |
    | Targeting | aws_subnet.main[1] | aws_subnet.main["subnet_a"] |
    | Partial updates | Risky | Safe |

    > Best practice: Use for_each when resources are not identical or when items may be removed from the middle of a collection.

    Conditionals

    Ternary Operator

    resource "aws_instance" "app" {
  count         = var.enabled ? 1 : 0
  instance_type = var.environment == "prod" ? "m5.xlarge" : "t3.micro"
}

    Conditional Blocks with count

    # Enable NAT Gateway only for non-dev environments
resource "aws_nat_gateway" "main" {
  count         = var.environment != "dev" ? 1 : 0
  allocation_id = aws_eip.nat[0].id
  subnet_id     = aws_subnet.public[0].id
}

    Conditional Expressions with Variables

    locals {
  # Select AMI based on environment
  ami_id = var.environment == "prod" ? data.aws_ami.prod.id : data.aws_ami.dev.id

    # Feature flags
      log_retention = var.environment == "prod" ? 90 : 7
    

    # Coalesce for defaults
      name = coalesce(var.custom_name, "${var.project}-${var.environment}")
    }

    Preconditions & Postconditions

    resource "aws_subnet" "main" {
  vpc_id     = var.vpc_id
  cidr_block = var.cidr_block

    lifecycle {
        precondition {
          condition     = tonumber(split("/", var.cidr_block)[1]) <= 24
          error_message = "CIDR block must be /24 or smaller."
        }
        postcondition {
          condition     = self.available_ip_address_count > 200
          error_message = "Subnet must have more than 200 available IPs."
        }
      }
    }

    Terraform Plan / Apply / Destroy Workflow

    Full Workflow

    # 1. Initialize — download providers, modules, initialize backend
terraform init

    2. Format — canonical formatting
    
terraform fmt -recursive

    3. Validate — syntax & type checking
    
terraform validate

    4. Plan — preview changes (dry run)
    
terraform plan -out=tfplan
terraform plan -var-file=prod.tfvars -out=tfplan

    5. Apply — execute changes
    
terraform apply tfplan              # Apply saved plan
terraform apply -auto-approve       # Skip confirmation (CI/CD only)

    6. Destroy — remove all resources
    
terraform destroy                   # Interactive
terraform destroy -auto-approve     # CI/CD only

    7. Refresh — sync state with real resources
    
terraform refresh                   # Deprecated in 1.x — use plan -refresh-only
terraform plan -refresh-only       # Preferred

    Plan Output Flags

    # Show full resource details
terraform plan -out=tfplan -detailed-exitcode

    Exit codes: 0 = no changes, 1 = error, 2 = changes present
    

    Target specific resources
    
terraform plan -target=aws_instance.web -target=aws_security_group.web

    Destroy plan
    
terraform plan -destroy

    JSON output for programmatic consumption
    
terraform plan -out=tfplan && terraform show -json tfplan > plan.json

    CI/CD Integration

    # In CI/CD pipelines (GitHub Actions example)
terraform init -backend-config="key=infra/$BRANCH/terraform.tfstate"
terraform plan -var-file="envs/$ENV.tfvars" -out=tfplan
terraform apply -auto-approve tfplan

    With destroy checking
    
terraform plan -destroy -var-file="envs/$ENV.tfvars"

    Import Existing Resources

    # Basic import
terraform import aws_instance.web i-0abcd1234efgh5678

    Import with module path
    
terraform import module.vpc.aws_subnet.private[0] subnet-0123456789abcdef0

    Import into for_each resource
    
terraform import aws_subnet.main["subnet_a"] subnet-0123456789abcdef0

    Import with count
    
terraform import aws_subnet.main[0] subnet-0123456789abcdef0

    Import Block (Terraform 1.5+)

    # Declarative import — can be committed to version control
import {
  to = aws_instance.web
  id = "i-0abcd1234efgh5678"
}

    import {
      to = aws_security_group.web
      id = "sg-0123456789abcdef0"
    }

    Generate Config from Import (Terraform 1.5+)

    # Generate HCL for an imported resource
terraform plan -generate-config-out=generated.tf

    Then review and refine generated.tf before applying

    State Manipulation — mv, rm, import

    # List all resources in state
terraform state list

    Show resource details
    
terraform state show aws_instance.web

    Move resource (refactor without destroying)
    
terraform state mv aws_instance.web aws_instance.app
terraform state mv module.old.module.vpc module.new.module.vpc

    Remove from state (keep real resource)
    
terraform state rm aws_instance.web

    Replace resource in state (force recreation on next apply)
    
terraform state replace-provider registry.terraform.io/hashicorp/aws registry.terraform.io/hashicorp/aws

    Pull state to local file
    
terraform state pull > state.json

    Push state from local file (force)
    
terraform state push state.json

    Moved Blocks (Declarative Refactoring)

    # In the OLD configuration location, add:
moved {
  from = aws_instance.web
  to   = aws_instance.app
}

    Terraform automatically detects the move during plan/apply. After all states are migrated, remove the moved block.

    Removed Blocks (Declarative De-provisioning)

    removed {
  from = aws_instance.old_web

    lifecycle {
        destroy = false  # Don't destroy the real resource
      }
    }

    Drift Detection

    Checking for Drift

    # Refresh state and show changes
terraform plan -detailed-exitcode

    Refresh-only mode (safe, no changes applied)
    
terraform plan -refresh-only

    Detect drift without planning changes
    
terraform plan -refresh-only -detailed-exitcode

    Drift Detection in CI/CD

    # GitHub Actions example

  • name: Drift Detection
    • 
  run: |
    terraform plan -detailed-exitcode -refresh-only
    EXIT_CODE=$?
    if [ $EXIT_CODE -eq 2 ]; then
      echo "::warning::Infrastructure drift detected!"
      exit 0  # Don't fail the pipeline, just warn
    elif [ $EXIT_CODE -eq 1 ]; then
      echo "Error during drift detection"
      exit 1
    else
      echo "No drift detected"
    fi

    Handling Drift

    # Option 1: Accept drift — update state to match reality
terraform apply -refresh-only

    Option 2: Correct drift — reapply config to match state
    
terraform apply

    Option 3: Ignore specific attributes
    
resource "aws_instance" "web" {
  # ...
  lifecycle {
    ignore_changes = [tags, user_data_replace_on_change]
  }
}

    Sentinel & OPA Policy

    Sentinel (Terraform Cloud/Enterprise)

    # Enforce tagging on all resources
import "tfplan/v2" as tfplan

    allResources = filter tfplan.resource_changes as addr, rc {
        rc.mode == "managed" and
        rc.type is not "data" and
        rc.change.actions is not ["delete"]
    }
    

    tagCheck = rule {
        all allResources as _, rc {
            rc.change.after.tags contains "Environment"
            rc.change.after.tags contains "CostCenter"
        }
    }
    

    main = rule {
        tagCheck
    }

    OPA (Open Policy Agent) — Works with any Terraform

    # policies/tags.rego
package terraform

    import future.keywords.in
    

    deny[msg] {
        resource := input.resource_changes[_]
        resource.mode == "managed"
        resource.type in {"aws_instance", "aws_rds_instance", "aws_s3_bucket"}
        not has_required_tags(resource.change.after.tags)
        msg := sprintf("Resource %s missing required tags", [resource.address])
    }
    

    has_required_tags(tags) {
        tags.Environment
        tags.CostCenter
        tags.ManagedBy
    }
    

    Enforce no public S3 buckets
    
deny[msg] {
    resource := input.resource_changes[_]
    resource.type == "aws_s3_bucket"
    resource.change.after.acl == "public-read"
    msg := sprintf("S3 bucket %s must not be public", [resource.address])
}

    Enforce instance types
    
deny[msg] {
    resource := input.resource_changes[_]
    resource.type == "aws_instance"
    resource.change.after.instance_type not in ["t3.micro", "t3.small", "t3.medium"]
    msg := sprintf("Instance %s uses unauthorized type: %s", [resource.address, resource.change.after.instance_type])
}

    Running OPA Checks

    # Generate plan JSON
terraform plan -out=tfplan
terraform show -json tfplan > plan.json

    Evaluate policies
    
opa eval --data policies/ --input plan.json "data.terraform.deny"

    With OPA CLI
    
opa eval -d policies/tags.rego -i plan.json "data.terraform.deny"

    Conftest (Docker-based OPA policy)

    # Check plan against policies
conftest test plan.json --policy policies/

    Workspaces Pattern

    Directory-Based Environment Pattern (Recommended)

    environments/
├── dev/
│   ├── main.tf          # module blocks only
│   ├── variables.tf
│   ├── dev.tfvars
│   └── backend.tf       # dev-specific backend config
├── staging/
│   ├── main.tf
│   ├── variables.tf
│   ├── staging.tfvars
│   └── backend.tf
└── prod/
    ├── main.tf
    ├── variables.tf
    ├── prod.tfvars
    └── backend.tf

    # environments/dev/main.tf
module "app" {
  source      = "../../modules/app"
  environment = "dev"
  instance    = "t3.micro"
  min_size    = 1
  max_size    = 2
}

    Workspace-Based Environment Pattern (Lighter)

    # Root module with workspace-aware config
main.tf
variables.tf
outputs.tf

    Per-workspace variable files
    
dev.tfvars
staging.tfvars
prod.tfvars

    # Variable-driven workspace pattern
locals {
  workspace_vars = {
    dev     = { instance = "t3.micro",  min = 1, max = 2 }
    staging = { instance = "t3.medium", min = 2, max = 4 }
    prod    = { instance = "t3.xlarge", min = 4, max = 10 }
  }
  config = local.workspace_vars[terraform.workspace]
}

    Pattern Comparison

    | Aspect | Directory Pattern | Workspace Pattern |
    |--------|-------------------|-------------------|
    | State isolation | Separate backends | Separate state files |
    | Configuration drift | Explicit per-env | Implicit per-workspace |
    | CI/CD integration | Simple — per dir | Needs workspace select |
    | Code duplication | Some (module calls) | Minimal |
    | Blast radius | Isolated | Shared code, shared risk |
    | Best for | Production | Simple environments |

    Flowcharts & Decision Diagrams

    Terraform Workflow Decision Tree

    Code Change → terraform fmt ✓ → terraform validate ✓ → terraform plan
                                                         │
                                              Changes detected? ──No──→ No action needed
                                                         │
                                                        Yes
                                                         ↓
                                              plan output reviewed? ──No──→ Review with team
                                                         │
                                                        Yes
                                                         ↓
                                          terraform apply (auto-approve? ──No──→ Interactive confirm)
                                                         │
                                                        Done
                                                         ↓
                                              State locked? ──Yes──→ Wait or force-unlock
                                                         │
                                                         ↓
                                              Remote state updated → Push to VCS

    State Management Decision Tree

    State issue? ─┬─ Drift detected → terraform plan -refresh-only → terraform apply
               ├─ Lock stuck → force-unlock  (dangerous, use with care)
               ├─ Resource moved → terraform state mv  
               ├─ Resource deleted outside → terraform import   → terraform apply
               └─ Corrupted state → restore from S3 versioning or backup

    Module Architecture Diagram

    ┌─────────────────────────────────────────────┐
│                Root Module                   │
│  (environment workspaces: dev/stg/prod)     │
├──────────┬──────────┬──────────┬────────────┤
│  VPC     │  RDS     │  ECS     │  S3/Bucket │
│  Module  │  Module  │  Module  │  Module     │
├──────────┴──────────┴──────────┴────────────┤
│          Remote State Data Sources           │
│  (vpc_id, subnet_ids, security_group_ids)    │
└─────────────────────────────────────────────┘

    Testing Strategy for Terraform

    Pre-Plan Validation

  • Format Check: terraform fmt -check -diff — ensures consistent style
  • Validate: terraform validate — catches syntax errors before plan
  • Lint with tflint:
    •    # Install tflint
   curl -s https://raw.githubusercontent.com/terraform-linters/tflint/master/install_linux.sh | bash
   tflint --init
   tflint --recursive  # Scan all modules
  • Security Scan with tfsec:
    •    # Install tfsec
   curl -sL https://raw.githubusercontent.com/aquasecurity/tfsec/master/scripts/install_linux.sh | bash
   tfsec . --format json --out tfsec-results.json
  • Cost Estimation with Infracost:
    •    # Install infracost
   curl -sL https://infracost.io/downloads/v0.10.sh | bash
   infracost breakdown --path=. --format=json --out-file=/tmp/infracost.json
   infracost diff --path=/tmp/infracost.json --compare-to=/tmp/infracost-prev.json

    Sentinel & OPA Policy Testing

    # Test Sentinel policies
sentinel test policy/*.sentinel

    Test OPA/Rego policies
    
opa test policies/ --verbose

    CI integration — fail on policy violation
    
conftest test terraform-plan.json --policy policies/

    Terraform Plan as Test Artifact

    # Generate plan JSON for automated review
terraform plan -out=tfplan
terraform show -json tfplan > plan.json

    Query plan for destructive changes
    
jq '[.resource_changes[] | select(.change.actions | contains(["delete"]))]' plan.json

    Verify no force-replace on production databases
    
jq '[.resource_changes[] | select(.type == "aws_db_instance" and .change.actions | contains(["delete"]))]' plan.json

    Terratest Integration (Go)

    package test

    import (
        "testing"
        "github.com/gruntwork-io/terratest/modules/terraform"
        "github.com/gruntwork-io/terratest/modules/aws"
        "github.com/stretchr/testify/assert"
    )
    

    func TestVpcModule(t *testing.T) {
        terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
            TerraformDir: "../modules/vpc",
            Vars: map[string]interface{}{
                "vpc_cidr": "10.0.0.0/16",
                "environment": "test",
            },
        })
        defer terraform.Destroy(t, terraformOptions)
        terraform.InitAndApply(t, terraformOptions)
        vpcID := terraform.Output(t, terraformOptions, "vpc_id")
        assert.NotEmpty(t, vpcID)
    }

    Performance Optimization for Terraform

  • Parallelism Tuning: terraform apply -parallelism=50 (default=10, increase for many independent resources)
  • Provider Cache: Use .terraform/providers cache across workspaces with plugin cache dir
    •    # ~/.terraformrc
   provider_installation {
     filesystem_mirror { path = "/usr/local/share/terraform/providers" }
     direct {}  # fallback to registry
   }
  • State File Optimization: Split large states into smaller modules
  • Targeted Apply for Speed: terraform apply -target=module.vpc during development
  • Refresh Skipping: terraform plan -refresh=false for faster plans when no remote changes expected
  • State Locking: Enable DynamoDB (AWS) or GCS (GCP) locking to prevent concurrent applies
  • Workspace Isolation: Use separate state storage per environment (s3://tf-state/dev, s3://tf-state/prod)

    • Best Practices

    Naming Conventions

    # Resources: snake_case, descriptive
resource "aws_security_group" "web_app" {}      # ✅
resource "aws_security_group" "sg1" {}          # ❌ Too terse

    Variables: snake_case, descriptive
    
variable "vpc_cidr_block" {}                    # ✅
variable "cidr" {}                              # ❌ Ambiguous

    Modules: snake_case
    
module "vpc" {}                                 # ✅

    Outputs: snake_case, descriptive
    
output "vpc_id" {}                              # ✅
output "id" {}                                  # ❌ Ambiguous

    Files per concern:
    

    main.tf, variables.tf, outputs.tf, versions.tf
    

    data.tf, locals.tf, providers.tf

    Tagging Strategy

    variable "mandatory_tags" {
  type = map(string)
  default = {
    ManagedBy = "terraform"
    Project   = "myapp"
  }
}

    locals {
      common_tags = merge(var.mandatory_tags, {
        Environment = var.environment
        CostCenter  = var.cost_center
        Owner       = var.owner
      })
    }
    

    resource "aws_instance" "web" {
      tags = merge(local.common_tags, {
        Name = "${local.name_prefix}-web"
      })
    }

    State Locking

  • Always use state locking in shared environments
  • S3 backend: Enable DynamoDB locking table
  • GCS: Built-in locking
  • Azure Blob: Built-in lease-based locking
  • Never terraform apply simultaneously on the same state

    • CI/CD Integration

    # .github/workflows/terraform.yml
name: Terraform CI/CD

    on:
      push:
        branches: [main]
      pull_request:
        branches: [main]
    

    jobs:
      terraform:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4
    

    - name: Setup Terraform
            uses: hashicorp/setup-terraform@v3
            with:
              terraform_version: '1.7.0'
    

    - name: Terraform Init
            run: terraform init -backend-config="key=$ENV/terraform.tfstate"
    

    - name: Terraform Format
            run: terraform fmt -check -recursive
    

    - name: Terraform Validate
            run: terraform validate
    

    - name: Terraform Plan
            run: terraform plan -var-file=envs/$ENV.tfvars -out=tfplan
    

    - name: Terraform Apply
            if: github.ref == 'refs/heads/main'
            run: terraform apply -auto-approve tfplan

    GitOps with Terraform

    # Atlantis (auto-plan on PR, apply on merge)

    .atlantis.yaml
    
repos:

  • id: /.*/
    • 
  apply_requirements: [approved, mergeable]
  workflow: default
  allowed_overrides: [workflow]

    Terragrunt (DRY remote state & provider config)
    

    terragrunt.hcl
    
remote_state {
  backend = "s3"
  config = {
    bucket = "terraform-state-${get_env("TF_VAR_env", "dev")}"
    key    = "${path_relative_to_include()}/terraform.tfstate"
    region = "us-east-1"
  }
}

    Security Best Practices

  • Never commit .terraform/ or terraform.tfstate to Git
  • Never commit secrets — use sensitive = true and secret managers
  • Always encrypt state files at rest (S3 + KMS, GCS + CMEK)
  • Restrict state file access via IAM policies
  • Use TF_VAR_ env vars for secrets in CI/CD
  • Rotate backend credentials regularly

    • .gitignore for Terraform

    # Local .terraform directories
.terraform/

    .tfstate files
    
*.tfstate
.tfstate.

    Crash log
    
crash.log
crash.*.log

    Exclude all .tfvars files
    
*.tfvars
*.tfvars.json

    Ignore override files
    
override.tf
override.tf.json
*_override.tf
*_override.tf.json

    Ignore CLI configuration
    
.terraformrc
terraform.rc

    Lock file (commit this!)
    

    .terraform.lock.hcl
    

    Sensitive files
    
ssh_key
*.pem
*.key

    Common Pitfalls

    1. State File Conflicts

    Error: Error acquiring the state lock
    Cause: Another process is running terraform apply. Fix: Ensure state locking is enabled. If stale lock: 
    terraform force-unlock 

    Only as a last resort!

    2. Resource Recreated on Every Apply

    Cause: Attributes that Terraform doesn't track are changing, or ignore_changes is missing. Fix: Add lifecycle { ignore_changes = [...] }.

    3. count/for_each Index Shifts

    Cause: Removing an item from the middle of a count list shifts all subsequent resources. Fix: Use for_each instead of count for heterogeneous collections.

    4. Provider Version Drift

    Cause: Team members running different provider versions. Fix: Pin provider and Terraform versions. Commit .terraform.lock.hcl.

    5. Sensitive Values in State

    Cause: sensitive = true only hides console output, not state file contents. Fix: Store secrets in a vault (AWS Secrets Manager, Vault) and reference via data sources.

    6. Circular Dependencies

    Cause: Resources referencing each other in a loop. Fix: Restructure modules, use depends_on, or split into separate applies.

    7. Large State Files

    Cause: Monolithic state files slow down operations. Fix: Split state by component (network, compute, database) using remote state data sources for cross-references.

    8. Orphaned Resources

    Cause: Removed from config but still in state, or created outside Terraform. Fix: terraform state rm to remove from state; terraform import to bring under management.

    9. Workspaces as Environments Anti-Pattern

    Cause: Using workspace names in complex logic for fundamentally different environments. Fix: Use directory-based separation for environments that differ significantly (e.g., different clouds, different architectures).

    10. Provisioner Dependency on Ephemeral State

    Cause: local-exec depending on tools available on the operator's machine. Fix: Use cloud-init, user data, or configuration management tools instead of provisioners.

    Quick Reference — Commands

    Initialization & Setup

    terraform init                              # Initialize working directory
terraform init -upgrade                    # Upgrade providers
terraform init -backend-config="key=dev/tfstate"  # Partial backend config
terraform init -reconfigure                 # Reconfigure backend (migration)
terraform get                               # Download/update modules
terraform version                           # Show Terraform version

    Planning & Applying

    terraform plan                              # Preview changes
terraform plan -out=tfplan                  # Save plan
terraform plan -destroy                     # Plan destruction
terraform plan -target=aws_instance.web     # Target specific resource
terraform plan -refresh-only                # Refresh state only
terraform plan -var-file=prod.tfvars        # Use variable file

    terraform apply                             # Apply changes (interactive)
    terraform apply tfplan                      # Apply saved plan
    terraform apply -auto-approve               # Skip confirmation
    terraform apply -target=aws_instance.web    # Apply specific resource
    

    terraform destroy                           # Destroy all
    terraform destroy -target=aws_instance.web  # Destroy specific resource
    terraform destroy -auto-approve             # Skip confirmation

    State Management

    terraform state list                        # List resources in state
terraform state show              # Show resource details
terraform state mv                # Move resource in state
terraform state rm                # Remove from state
terraform state pull                        # Pull state to stdout
terraform state push                  # Push state from file
terraform import              # Import existing resource
terraform force-unlock             # Force unlock state

    Workspace Commands

    terraform workspace list                    # List workspaces
terraform workspace show                   # Show current workspace
terraform workspace new               # Create workspace
terraform workspace select            # Switch workspace
terraform workspace delete            # Delete workspace

    Formatting & Validation

    terraform fmt                               # Format files
terraform fmt -recursive                    # Format all files recursively
terraform fmt -check                        # Check formatting (CI)
terraform validate                          # Validate configuration
terraform validate -json                    # JSON output

    Debugging

    TF_LOG=DEBUG terraform apply                # Debug logging
TF_LOG=TRACE terraform plan                 # Trace-level logging
TF_LOG_PATH=./terraform.log terraform apply # Log to file

    Graph visualization
    
terraform graph | dot -Tpng > graph.png     # Generate dependency graph
terraform graph -draw-cycles                # Show cycles

    Output commands
    
terraform output                           # Show all outputs
terraform output vpc_id                    # Show specific output
terraform output -json                     # JSON output
terraform output -raw vpc_id               # Raw string output

    Testing

    # Terraform test (native, 1.6+)
terraform test                              # Run test files

    terraform validate
    
terraform validate                         # Syntax & type checking

    third-party tools
    
tflint                                     # Lint Terraform files
checkov -d .                               # Security scanning
tfsec                                     # Security scanning
terrascan scan -t aws                      # Compliance scanning

    Pulumi Quick Reference (Cross-Reference)

    pulumi new aws-typescript                  # Create new project
pulumi up                                 # Deploy changes
pulumi preview                            # Preview changes
pulumi destroy                            # Remove all resources
pulumi stack ls                           # List stacks
pulumi stack select prod                  # Switch stack
pulumi import aws:ec2/instance:Instance web i-1234  # Import
pulumi state delete urn:pulumi:dev...     # Remove from state

    CloudFormation Quick Reference (Cross-Reference)

    aws cloudformation create-stack \
  --stack-name my-stack \
  --template-body file://template.yaml \
  --parameters ParameterKey=Env,ParameterValue=prod

    aws cloudformation update-stack \
      --stack-name my-stack \
      --template-body file://template.yaml
    

    aws cloudformation delete-stack --stack-name my-stack
    aws cloudformation describe-stacks --stack-name my-stack
    aws cloudformation list-exports