Terraform State: Store Terraform State File in S3 Bucket with DynamoDB Locking

Table of Contents

Prerequisites
#

Terraform Installation (Deb)
#

# Install the HashiCorp GPG key
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg > /dev/null

# Verify the GPG key fingerprint
gpg --no-default-keyring --keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg --fingerprint

# Add the official HashiCorp repository 
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list

# Install Terraform
sudo apt update && sudo apt-get install terraform

# Verify installation / check version
terraform version

AWS CLI Installation
#

# Update packages
sudo apt update

# Unstall zip tool
sudo apt install unzip -y

# Download AWS CLI zip file
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

# Unzip
unzip awscliv2.zip

# Install
sudo ./aws/install

# Verify installation / check version
/usr/local/bin/aws --version

# Start AWS CLI configuration
aws configure

Terraform State S3 Setup
#

Create S3 Bucket
#

# Create a new S3 bucket in "eu-central-1"
aws s3api create-bucket \
    --bucket jkw-terraform-example \
    --region eu-central-1 \
    --create-bucket-configuration LocationConstraint=eu-central-1

# Shell output:
{
    "Location": "http://jkw-terraform-example.s3.amazonaws.com/"
}

Enable Bucket Versioning
#

S3 Versioning: Is a feature that keeps multiple versions of an object (for example the terraform.tfstate file) instead of overwriting it.

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket jkw-terraform-example \
  --versioning-configuration Status=Enabled \
  --region eu-central-1

Create DynamoDB Table
#

DynamoDB Table for State Locking: DynamoDB is used to prevent multiple people or processes from modifying Terraform state at the same time, which could corrupt the state file. When Terraform runs, it creates a lock record in DynamoDB, Other Terraform processes check for this lock and wait if another operation is in progress. After the Terraform process is done, it removes the lock.

# Create a DynamoDB table
aws dynamodb create-table --table-name terraform-locks \
  --attribute-definitions AttributeName=LockID,AttributeType=S \
  --key-schema AttributeName=LockID,KeyType=HASH \
  --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 \
  --region eu-central-1

# Shell output:
{
    "TableDescription": {
        "AttributeDefinitions": [
            {
                "AttributeName": "LockID",
                "AttributeType": "S"
            }
        ],
        "TableName": "terraform-locks",
        "KeySchema": [
            {
                "AttributeName": "LockID",
                "KeyType": "HASH"
            }
        ],
        "TableStatus": "CREATING",
        "CreationDateTime": "2025-01-29T10:42:45.929000+00:00",
        "ProvisionedThroughput": {
            "NumberOfDecreasesToday": 0,
            "ReadCapacityUnits": 1,
            "WriteCapacityUnits": 1
        },
        "TableSizeBytes": 0,
        "ItemCount": 0,
        "TableArn": "arn:aws:dynamodb:eu-central-1:012345678912:table/terraform-locks",
        "TableId": "1025b793-84c6-4a69-bd5e-8e2c01c51f7c",
        "DeletionProtectionEnabled": false
    }
}

Create IAM User
#

# Create a new IAM user
aws iam create-user --user-name terraform-s3-user

# Shell output:
{
    "User": {
        "Path": "/",
        "UserName": "terraform-s3-user",
        "UserId": "AIDARCHUALINQRHS7OLMC",
        "Arn": "arn:aws:iam::012345678912:user/terraform-s3-user",
        "CreateDate": "2025-01-29T10:43:39+00:00"
    }
}

Create & Attach IAM Policy
#

Create an IAM policy, that allows to access the S3 Bucket and the DynamoDB table:

terraform-s3-policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::jkw-terraform-example"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::jkw-terraform-example/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "dynamodb:GetItem",
                "dynamodb:PutItem",
                "dynamodb:DeleteItem",
                "dynamodb:Scan"
            ],
            "Resource": "arn:aws:dynamodb:eu-central-1:012345678912:table/terraform-locks"
        }
    ]
}

# Create IAM policy for S3 bucket access
aws iam create-policy --policy-name terraform-s3-policy --policy-document file://terraform-s3-policy.json

# Shell output:
{
    "Policy": {
        "PolicyName": "terraform-s3-policy",
        "PolicyId": "ANPARCHUALINR3XDUCVDU",
        "Arn": "arn:aws:iam::012345678912:policy/terraform-s3-policy",
        "Path": "/",
        "DefaultVersionId": "v1",
        "AttachmentCount": 0,
        "PermissionsBoundaryUsageCount": 0,
        "IsAttachable": true,
        "CreateDate": "2025-01-29T10:44:46+00:00",
        "UpdateDate": "2025-01-29T10:44:46+00:00"
    }
}

# Get policy ARN (Also available in the previous output)
aws iam list-policies --query "Policies[?PolicyName=='terraform-s3-policy'].Arn" --output text

# Shell output:
arn:aws:iam::012345678912:policy/terraform-s3-policy

# Attach policy to IAM user: Acess S3 Bucket & DynamoDB
aws iam attach-user-policy --user-name terraform-s3-user --policy-arn arn:aws:iam::012345678912:policy/terraform-s3-policy

# Attach policy to IAM user: EC2 FullAccess (for testing purposes)
aws iam attach-user-policy --user-name terraform-s3-user --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess

Create Access Keys
#

# Create Access Keys for the IAM user
aws iam create-access-key --user-name terraform-s3-user

# Shell output:
{
    "AccessKey": {
        "UserName": "terraform-s3-user",
        "AccessKeyId": "AKIARCHUALIN6MQUAQFK",
        "Status": "Active",
        "SecretAccessKey": "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",
        "CreateDate": "2025-01-29T10:47:15+00:00"
    }
}

Add AWS Credentials
#

Configure the AWS CLI and add the IAM user access keys:

# Start AWS CLI configuration
aws configure

# Verify credentials
cat ~/.aws/credentials

Terraform Example Project
#

Terraform Configuration Files
#

Project Folder
#

# Create a folder for the Terraform project
TF_PROJECT_NAME=terraform-example-project
mkdir $TF_PROJECT_NAME && cd $TF_PROJECT_NAME

Terraform State & Provider Configuration
#

terraform.tf

# Terraform State
terraform {
  backend "s3" {
    bucket         = "jkw-terraform-example" # Define S3 bucket
    key            = "terraform-example-project/state.tfstate" # Define Terraform state directory
    region         = "eu-central-1"
    dynamodb_table = "terraform-locks" # Define DynamoDB table
    encrypt        = true
  }
}

# Terraform Provider
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# Provider for AWS Region
provider "aws" {
  alias  = "aws_region"
  region = var.aws_region
}

AWS Example Resource (EC2 Instance)
#

This Terraform configuration creates an EC2 instance for testing purposes.

main.tf

## AWS Region
variable "aws_region" {
  description = "AWS Region"
  type        = string
  default     = "us-east-1"
}

# EC2 Image ID
variable "ami_id" {
  default = "ami-0e2c8caa4b6378d8c" # Define EC2 AMI ID
}

# EC2 Instance
resource "aws_instance" "ec2" {
  provider = aws.aws_region
  ami                    = var.ami_id
  instance_type          = "t2.micro"

  tags = {
    Name = "Terraform-Test-Instance"
  }
}

Apply Configuration
#

# Initialize the Terraform project
terraform init

# Validates the syntax and structure of Terraform configuration files
terraform validate

# Dry run / preview changes before applying them
terraform plan

# Create Terraform stack: Auto approve
terraform apply -auto-approve

Verify Terraform State File
#

# List files
aws s3 ls s3://jkw-terraform-example/terraform-example-project/

# Shell output:
2025-01-29 11:14:12       4908 state.tfstate

Cleanup
#

Delete DynamoDB Table
#

# Delete the DynamoDB table
aws dynamodb delete-table \
  --table-name terraform-locks \
  --region eu-central-1

# Verify deletion
aws dynamodb list-tables --region <wrong-region>

Delete S3 Bucket
#

# Delete all objects and their versions
aws s3api list-object-versions --bucket jkw-terraform-example --query 'Versions[].{Key:Key,VersionId:VersionId}' --output json | jq -c '.[]' | while read i; do
  key=$(echo $i | jq -r '.Key')
  version=$(echo $i | jq -r '.VersionId')
  aws s3api delete-object --bucket jkw-terraform-example --key "$key" --version-id "$version"
done

# Delete all markers
aws s3api list-object-versions --bucket jkw-terraform-example --query 'DeleteMarkers[].{Key:Key,VersionId:VersionId}' --output json | jq -c '.[]' | while read i; do
  key=$(echo $i | jq -r '.Key')
  version=$(echo $i | jq -r '.VersionId')
  aws s3api delete-object --bucket jkw-terraform-example --key "$key" --version-id "$version"
done

# Delete S3 Bucket
aws s3api delete-bucket \
  --bucket jkw-terraform-example \
  --region eu-central-1

Delete IAM User
#

List Attached Policies
#

# List policies that are attached to the S3 user
aws iam list-attached-user-policies --user-name terraform-s3-user

# Shell output:
{
    "AttachedPolicies": [
        {
            "PolicyName": "terraform-s3-policy",
            "PolicyArn": "arn:aws:iam::012345678912:policy/terraform-s3-policy"
        },
        {
            "PolicyName": "AmazonEC2FullAccess",
            "PolicyArn": "arn:aws:iam::aws:policy/AmazonEC2FullAccess"
        }
    ]
}

Detach Policies
#

# Detach "AmazonEC2FullAccess" policy
aws iam detach-user-policy --user-name terraform-s3-user --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess

# Detach "terraform-s3-policy" policy
aws iam detach-user-policy --user-name terraform-s3-user --policy-arn arn:aws:iam::012345678912:policy/terraform-s3-policy

Delete Custom Policy
#

# Delete the "terraform-s3-policy" policy
aws iam delete-policy --policy-arn arn:aws:iam::012345678912:policy/terraform-s3-policy

Delete Access Keys
#

# List access keys
aws iam list-access-keys --user-name terraform-s3-user

# Shell output:
{
    "AccessKeyMetadata": [
        {
            "UserName": "terraform-s3-user",
            "AccessKeyId": "AKIARCHUALIN6MQUAQFK",
            "Status": "Active",
            "CreateDate": "2025-01-29T10:47:15+00:00"
        }
    ]
}

# Delete the access key
aws iam delete-access-key --user-name terraform-s3-user --access-key-id AKIARCHUALIN6MQUAQFK

Delete the IAM User
#

# Delete the IAM user
aws iam list-users --query "Users[?UserName=='terraform-s3-user']"

Prerequisites #

Terraform Installation (Deb) #

AWS CLI Installation #

Terraform State S3 Setup #

Create S3 Bucket #

Enable Bucket Versioning #

Create DynamoDB Table #

Create IAM User #

Create & Attach IAM Policy #

Create Access Keys #

Add AWS Credentials #

Terraform Example Project #

Terraform Configuration Files #

Project Folder #

Terraform State & Provider Configuration #

AWS Example Resource (EC2 Instance) #

Apply Configuration #

Verify Terraform State File #

Cleanup #

Delete DynamoDB Table #

Delete S3 Bucket #

Delete IAM User #

List Attached Policies #

Detach Policies #

Delete Custom Policy #

Delete Access Keys #

Delete the IAM User #