GitHub Repository Available
Overview #
VPC & Subnet Routing Flowchart #
graph TD
%% VPC
subgraph VPC["VPC: 10.10.0.0/16"]
IGW["Internet Gateway"]
NAT1["NAT Gateway 1
Public Subnet
us-east-1a"] NAT2["NAT Gateway 2
Public Subnet
us-east-1b"] %% Routing Tables subgraph RoutingTables["Routing Tables"] PublicRT["Public Routing Table"] PrivateRT1["Private Routing Table 1"] PrivateRT2["Private Routing Table 2"] end %% Subnets subgraph Subnets["Subnets"] Subnet1["Public Subnet1 10.10.0.0/24
us-east-1a"] Subnet2["Public Subnet2 10.10.1.0/24
us-east-1b"] Subnet3["Private Subnet1 10.10.2.0/24
us-east-1a"] Subnet4["Private Subnet2 10.10.3.0/24
us-east-1b"] end PublicRT -->|Route: 0.0.0.0/0 to IGW| IGW PrivateRT1 -->|Route: 0.0.0.0/0 to NAT| NAT1 PrivateRT2 -->|Route: 0.0.0.0/0 to NAT| NAT2 Subnet1 -->|Associated| PublicRT Subnet2 -->|Associated| PublicRT Subnet3 -->|Associated| PrivateRT1 Subnet4 -->|Associated| PrivateRT2 NAT1 -->|Outbound Internet Traffic| IGW NAT2 -->|Outbound Internet Traffic| IGW end %% Dashed Border classDef BoarderDash stroke-dasharray:5 5; class Subnets,RoutingTables BoarderDash %% Node Colors style Subnet1 fill:#d1f7d6,stroke:#333,stroke-width:2px style Subnet2 fill:#d1f7d6,stroke:#333,stroke-width:2px style Subnet3 fill:#f7d6d6,stroke:#333,stroke-width:2px style Subnet4 fill:#f7d6d6,stroke:#333,stroke-width:2px style PublicRT fill:#d1f7d6,stroke:#333,stroke-width:2px style PrivateRT1 fill:#f7d6d6,stroke:#333,stroke-width:2px style PrivateRT2 fill:#f7d6d6,stroke:#333,stroke-width:2px style NAT1 fill:#d6f7f7,stroke:#333,stroke-width:2px style NAT2 fill:#d6f7f7,stroke:#333,stroke-width:2px
Public Subnet
us-east-1a"] NAT2["NAT Gateway 2
Public Subnet
us-east-1b"] %% Routing Tables subgraph RoutingTables["Routing Tables"] PublicRT["Public Routing Table"] PrivateRT1["Private Routing Table 1"] PrivateRT2["Private Routing Table 2"] end %% Subnets subgraph Subnets["Subnets"] Subnet1["Public Subnet1 10.10.0.0/24
us-east-1a"] Subnet2["Public Subnet2 10.10.1.0/24
us-east-1b"] Subnet3["Private Subnet1 10.10.2.0/24
us-east-1a"] Subnet4["Private Subnet2 10.10.3.0/24
us-east-1b"] end PublicRT -->|Route: 0.0.0.0/0 to IGW| IGW PrivateRT1 -->|Route: 0.0.0.0/0 to NAT| NAT1 PrivateRT2 -->|Route: 0.0.0.0/0 to NAT| NAT2 Subnet1 -->|Associated| PublicRT Subnet2 -->|Associated| PublicRT Subnet3 -->|Associated| PrivateRT1 Subnet4 -->|Associated| PrivateRT2 NAT1 -->|Outbound Internet Traffic| IGW NAT2 -->|Outbound Internet Traffic| IGW end %% Dashed Border classDef BoarderDash stroke-dasharray:5 5; class Subnets,RoutingTables BoarderDash %% Node Colors style Subnet1 fill:#d1f7d6,stroke:#333,stroke-width:2px style Subnet2 fill:#d1f7d6,stroke:#333,stroke-width:2px style Subnet3 fill:#f7d6d6,stroke:#333,stroke-width:2px style Subnet4 fill:#f7d6d6,stroke:#333,stroke-width:2px style PublicRT fill:#d1f7d6,stroke:#333,stroke-width:2px style PrivateRT1 fill:#f7d6d6,stroke:#333,stroke-width:2px style PrivateRT2 fill:#f7d6d6,stroke:#333,stroke-width:2px style NAT1 fill:#d6f7f7,stroke:#333,stroke-width:2px style NAT2 fill:#d6f7f7,stroke:#333,stroke-width:2px
RDS Postgres Flowchart #
graph TD
%% VPC
subgraph VPC["VPC: 10.10.0.0/16"]
%% Subnet 1
subgraph Subnet1["Public Subnet1"]
EC2-1["EC2 Instance"]
end
%% Subnet 2
subgraph Subnet2["Public Subnet2"]
EC2-2["EC2 Instance"]
end
%% RDS Subnet Group
subgraph SubnetGroup["RDS Subnet Group"]
%% Subnet 3
subgraph Subnet3["Private Subnet1"]
Postgres1["Postgres Node 1"]
end
%% Subnet 4
subgraph Subnet4["Private Subnet2"]
Postgres2["Postgres Node 2"]
end
%% Postgres HA
Postgres1 <-.->|HA| Postgres2
PostgresEP["Postgres Endpoint DNS"] -.-> Postgres1
PostgresEP -.-> Postgres2
end
EC2-1 -->|Allow outbound TCP 5432| SG-EC2
EC2-2 -->|Allow outbound TCP 5432| SG-EC2
SG-EC2["SG EC2"] -.->|Allow inbound TCP 5432
from SG EC2| SG-RDS["SG RDS"] SG-RDS --> PostgresEP end %% Dashed Border classDef BoarderDash stroke-dasharray:5 5; class Subnet1,Subnet2,Subnet3,Subnet4,SubnetGroup BoarderDash %% Node Colors style EC2-1 fill:#d1f7d6,stroke:#333,stroke-width:2px style EC2-2 fill:#d1f7d6,stroke:#333,stroke-width:2px style Postgres1 fill:#f7d6d6,stroke:#333,stroke-width:2px style Postgres2 fill:#f7d6d6,stroke:#333,stroke-width:2px style SG-EC2 fill:#d6f7f7,stroke:#333,stroke-width:2px style SG-RDS fill:#d6f7f7,stroke:#333,stroke-width:2px
from SG EC2| SG-RDS["SG RDS"] SG-RDS --> PostgresEP end %% Dashed Border classDef BoarderDash stroke-dasharray:5 5; class Subnet1,Subnet2,Subnet3,Subnet4,SubnetGroup BoarderDash %% Node Colors style EC2-1 fill:#d1f7d6,stroke:#333,stroke-width:2px style EC2-2 fill:#d1f7d6,stroke:#333,stroke-width:2px style Postgres1 fill:#f7d6d6,stroke:#333,stroke-width:2px style Postgres2 fill:#f7d6d6,stroke:#333,stroke-width:2px style SG-EC2 fill:#d6f7f7,stroke:#333,stroke-width:2px style SG-RDS fill:#d6f7f7,stroke:#333,stroke-width:2px
Prerequisites #
Install AWS CLI #
# Update packages
sudo apt update
# Unstall zip tool
sudo apt install unzip -y
# Download AWS CLI zip file
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
# Unzip
unzip awscliv2.zip
# Install
sudo ./aws/install
# Verify installation / check version
/usr/local/bin/aws --version
Configure AWS CLI #
# Start AWS CLI configuration
aws configure
Install Terraform #
# Install the HashiCorp GPG key
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg > /dev/null
# Verify the GPG key fingerprint
gpg --no-default-keyring --keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg --fingerprint
# Add the official HashiCorp repository
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
# Install Terraform
sudo apt update && sudo apt-get install terraform
# Verify installation / check version
terraform version
Terraform RDS Postgres Stack #
File and Folder Structure #
The file and folder structure of the Terraform project looks like this:
aws-rds-postgres-production-vpc
├── ec2.tf
├── outputs.tf
├── rds-postgres.tf
├── terraform.tf
├── variables.tf
└── vpc.tf
Terraform Configuration Files #
Project Folder & Terraform Provider #
# Create Terraform project folder
TF_PROJECT_NAME=aws-rds-postgres-production-vpc
mkdir $TF_PROJECT_NAME && cd $TF_PROJECT_NAME
- terraform.tf
# Terraform Provider
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
# Provider AWS Region
provider "aws" {
alias = "aws_region"
region = var.aws_region
}
Variables #
- variables.tf
# AWS Region
variable "aws_region" {
description = "AWS Region"
type = string
default = "us-east-1"
}
# VPC Name
variable "vpc_name" {
description = "The name of the VPC"
type = string
default = "Example-VPC"
}
## Postgres
# Postgres User
variable "postgres_user" {
description = "Postgres default user"
type = string
default = "postgres"
}
# Postgres Password
variable "postgres_pw" {
description = "Postgres default user"
type = string
default = "my-secure-pw"
}
## VPC & Subnets
# VPC CIDR
variable "vpc_cidr" {
description = "VPC CIDR block"
type = string
default = "10.10.0.0/16"
}
# Public Subnets CIDR (Loop)
variable "subnets_public_cidr" {
type = list(string)
default = ["10.10.0.0/24", "10.10.1.0/24"]
}
# Public Subnets AZ (Loop)
variable "subnets_public_azs" {
type = list(string)
default = ["us-east-1a", "us-east-1b"]
}
# Private Subnets CIDR (Loop)
variable "subnets_private_cidr" {
type = list(string)
default = ["10.10.2.0/24", "10.10.3.0/24"]
}
# Private Subnets AZ (Loop)
variable "subnets_private_azs" {
type = list(string)
default = ["us-east-1a", "us-east-1b"]
}
## EC2 Instances
# SSH key pair name
variable "key_name" {
default = "us-east-1-pc-le" # Define key pair name
}
# EC2 Image ID
variable "ami_id" {
default = "ami-0e2c8caa4b6378d8c" # Define EC2 AMI ID
}
VPC and Subnets #
- vpc.tf
# VPC "10.10.0.0/16"
resource "aws_vpc" "vpc" {
provider = aws.aws_region
cidr_block = var.vpc_cidr
enable_dns_support = true
enable_dns_hostnames = true
tags = {
Name = "${var.vpc_name}"
Env = "Production"
}
}
## Subnets
# Public Subnets (Loop)
resource "aws_subnet" "subnets_public" {
provider = aws.aws_region
count = length(var.subnets_public_cidr)
vpc_id = aws_vpc.vpc.id
cidr_block = element(var.subnets_public_cidr, count.index)
availability_zone = element(var.subnets_public_azs, count.index)
map_public_ip_on_launch = true
tags = {
Name = "${var.vpc_name} Subnet-Public-${count.index + 1}"
Env = "Production"
}
}
# Private Subnets (Loop)
resource "aws_subnet" "subnets_private" {
provider = aws.aws_region
count = length(var.subnets_private_cidr)
vpc_id = aws_vpc.vpc.id
cidr_block = element(var.subnets_private_cidr, count.index)
availability_zone = element(var.subnets_private_azs, count.index)
tags = {
Name = "${var.vpc_name} Subnet-Private-${count.index + 1}"
Env = "Production"
}
}
## Gateways
# Internet Gateway
resource "aws_internet_gateway" "vpc_igw" {
provider = aws.aws_region
vpc_id = aws_vpc.vpc.id
depends_on = [aws_vpc.vpc]
tags = {
Name = "${var.vpc_name} IGW"
Env = "Production"
}
}
# Elastic IPs (EIP) for NAT Gateways (Loop)
resource "aws_eip" "nat_gw_eip" {
provider = aws.aws_region
count = length(var.subnets_public_cidr)
domain = "vpc"
tags = {
Name = "VPC NAT-GW EIP-${count.index + 1}"
Env = "Production"
}
}
# NAT Gateways (Loop)
resource "aws_nat_gateway" "nat_gw" {
provider = aws.aws_region
count = length(var.subnets_public_cidr)
allocation_id = element(aws_eip.nat_gw_eip[*].id, count.index)
subnet_id = aws_subnet.subnets_public[count.index].id
depends_on = [aws_internet_gateway.vpc_igw]
tags = {
Name = "VPC NAT-GW-${count.index + 1}"
Env = "Production"
}
}
## Routing
# Public Route Table
resource "aws_route_table" "public_rt" {
provider = aws.aws_region
vpc_id = aws_vpc.vpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.vpc_igw.id
}
depends_on = [
aws_vpc.vpc,
aws_internet_gateway.vpc_igw
]
tags = {
Name = "${var.vpc_name} Public Route Table"
Env = "Production"
}
}
# Private Route Tables (Loop)
resource "aws_route_table" "private_rt" {
provider = aws.aws_region
count = length(var.subnets_private_cidr)
vpc_id = aws_vpc.vpc.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = element(aws_nat_gateway.nat_gw[*].id, count.index)
}
tags = {
Name = "${var.vpc_name} Private Route Table-${count.index + 1}"
Env = "Production"
}
}
# Associate Route Tables with Public Subnets
resource "aws_route_table_association" "subnets_public_ra" {
provider = aws.aws_region
count = length(var.subnets_public_cidr)
subnet_id = element(aws_subnet.subnets_public.*.id, count.index)
route_table_id = aws_route_table.public_rt.id
depends_on = [aws_route_table.public_rt]
}
# Associate Route Tables with Private Subnets
resource "aws_route_table_association" "private_rta" {
provider = aws.aws_region
count = length(var.subnets_private_cidr)
subnet_id = aws_subnet.subnets_private[count.index].id
route_table_id = aws_route_table.private_rt[count.index].id
}
RDS PostgreSQL #
- rds-postgres.tf
# RDS Subnet Group
resource "aws_db_subnet_group" "rds_subnet_group" {
provider = aws.aws_region
name = "main"
subnet_ids = [aws_subnet.subnets_private[0].id, aws_subnet.subnets_private[1].id]
depends_on = [aws_subnet.subnets_private]
tags = {
Name = "${var.vpc_name} RDS Subnet Group"
Env = "Production"
}
}
# RDS PostgreSQL
resource "aws_db_instance" "rds_db" {
provider = aws.aws_region
allocated_storage = 10
identifier = "postgres-test2"
db_subnet_group_name = aws_db_subnet_group.rds_subnet_group.id
engine = "postgres"
engine_version = "14"
instance_class = "db.t3.micro"
username = var.postgres_user
password = var.postgres_pw
vpc_security_group_ids = [aws_security_group.sg_postgres.id]
skip_final_snapshot = true
multi_az = true # Enables Multi-AZ for HA
depends_on = [aws_db_subnet_group.rds_subnet_group]
}
# RDS Security Group
resource "aws_security_group" "sg_postgres" {
provider = aws.aws_region
name = "SG_Allow_Postgres"
description = "Allow Postgres traffic"
vpc_id = aws_vpc.vpc.id
ingress {
description = "Allow Postgres from EC2 instances"
from_port = 5432
to_port = 5432
protocol = "tcp"
security_groups = [aws_security_group.sg_ec2.id] # EC2 security group
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
depends_on = [aws_vpc.vpc]
tags = {
Name = "${var.vpc_name} RDS Postgres"
Env = "Production"
}
}
EC2 Instances #
- ec2.tf
# EC2 Instances in Public Subnets (Loop)
resource "aws_instance" "vm" {
provider = aws.aws_region
count = length(var.subnets_public_cidr) # Create as many instances as subnets
ami = var.ami_id
instance_type = "t3.small"
subnet_id = aws_subnet.subnets_public[count.index].id
key_name = var.key_name
vpc_security_group_ids = [aws_security_group.sg_ec2.id]
depends_on = [
aws_vpc.vpc,
aws_security_group.sg_ec2
]
tags = {
Name = "VM Public Subnet ${count.index + 1}"
Env = "Production"
}
}
# Security Group for SSH Access and Ping
resource "aws_security_group" "sg_ec2" {
provider = aws.aws_region
name = "SG"
description = "Security group for SSH access and ping"
vpc_id = aws_vpc.vpc.id
ingress {
description = "Allow SSH from anywhere"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
description = "Allow ping"
from_port = 8
to_port = -1
protocol = "icmp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
description = "Allow all outbound traffic"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
depends_on = [aws_vpc.vpc]
tags = {
Name = "SG"
Env = "Production"
}
}
Output #
- outputs.tf
# RDS Postgres DNS Endpoints
output "rds_endpoint" {
value = aws_db_instance.rds_db.endpoint
}
# EC2 Public IPs
output "EC2_Public_IPs" {
description = "Public IPs of the EC2 instances"
value = aws_instance.vm[*].public_ip
}
# EC2 Private IPs
output "EC2_Private_IPs" {
description = "Private IPs of the EC2 instances"
value = aws_instance.vm[*].private_ip
}
Configuration Deployment #
Initialize Terraform Project #
This will download and install the AWS Terraform provider defined in the terraform.tf file with “hashicorp/aws”, as well as setting up the configuration files in the project directory.
# Initialize the Terraform project
terraform init
Validate Configuration Files #
# Validates the syntax and structure of Terraform configuration files
terraform validate
# Shell output:
Success! The configuration is valid.
Plan the Deployment #
# Dry run / preview changes before applying them
terraform plan
Apply the Configuration #
Note that it takes a while for the setup to complete:
aws_db_instance.rds_db: Creation complete after 11m58s [id=db-MC5IDNZUJMRAAXQYBGPBV2N2I4]
# Create network stack
terraform apply -auto-approve
# Shell output:
Apply complete! Resources: 23 added, 0 changed, 0 destroyed.
Outputs:
EC2_Private_IPs = [
"10.10.0.225",
"10.10.1.32",
]
EC2_Public_IPs = [
"54.159.73.95",
"44.192.117.68",
]
rds_endpoint = "postgres-test2.cpyy8k66ctzh.us-east-1.rds.amazonaws.com:5432"
RDS Endpoint:
-
RDS provides a single DNS endpoint regardless of whether the database is deployed in a Single-AZ or Multi-AZ configuration
-
This endpoint automatically resolves to the primary instance (writer) in your RDS setup.
Verify PostgreSQL Connectivity #
SSH Into EC2 Instance #
# SSH into EC2 instance 1
ssh -i /home/ubuntu/.ssh/us-east-1-pc-le.pem ubuntu@54.159.73.95
# SSH into EC2 instance 2
ssh -i /home/ubuntu/.ssh/us-east-1-pc-le.pem ubuntu@44.192.117.68
Install Postgres Client #
# Install the "postgresql-common" package, used for the helper scripts
sudo apt install postgresql-common -y
# Add the PostgreSQL (PGDG) repository for latest PostgreSQL versions (bypass installation prompt)
yes "" | sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
# Install Postgres client
sudo apt install postgresql-client -y
Connect to PostgreSQL Server #
# Connect to PostgreSQL
psql -h postgres-test2.cpyy8k66ctzh.us-east-1.rds.amazonaws.com -U postgres -d postgres
# Shell output
Password for user postgres: # Enter PW
-
-U postgres
Username to authenticate with the PostgreSQL database -
-d postgres
Database name to connect to
# List Postgres version:
SELECT version();
# Shell output:
version
----------------------------------------------------------------------------------------------------------
PostgreSQL 14.12 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-17), 64-bit
(1 row)
# Exit Postgres
\q
Links #
# GitHub Repository
https://github.com/jueklu/terraform-aws-rds-postgres