Overview #
My Cluster #
In this tutorial I’m using the following K3s Kubernetes cluster:
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ubuntu1 Ready control-plane,master 2d13h v1.30.5+k3s1 192.168.30.10 <none> Ubuntu 24.04.1 LTS 6.8.0-45-generic containerd://1.7.21-k3s2
ubuntu2 Ready worker 2d13h v1.30.5+k3s1 192.168.30.11 <none> Ubuntu 24.04.1 LTS 6.8.0-45-generic containerd://1.7.21-k3s2
ubuntu3 Ready worker 2d13h v1.30.5+k3s1 192.168.30.12 <none> Ubuntu 24.04.1 LTS 6.8.0-45-generic containerd://1.7.21-k3s2
ubuntu4 Ready worker 2d13h v1.30.5+k3s1 192.168.30.13 <none> Ubuntu 24.04.1 LTS 6.8.0-45-generic containerd://1.7.21-k3s2
Kustomize Summary #
-
Kustomize is used for customizing configurations of Kubernetes resources / deployments.
-
Overlays are applied on top of a base configurations like Deployments, Services, ConfigMaps,…
-
It allows to manage different environments (dev, production, project1, project2,…) by applying these overlays to a base configuration.
-
Unlike tools like Helm, Kustomize doesn’t use templates. It works by merging YAML files.
-
Kustomize relies on directories to organize files. Each environment has its own folder with a kustomization.yaml file that lists the base files and any overlays or patches.
Main Kustomize Commands #
# View merged manifests
kubectl kustomize /path/to/kustomization
# Apply the Kustomize configuration from the specified directory
kubectl apply -k /path/to/kustomization
Kustomize Example #
Create Namespaces #
# Create namespaces for the overlay deployments
kubectl create ns nginx-dev
kubectl create ns nginx-prod
Folder Structure #
# Create the folder structure
mkdir -p nginx-example/{base,overlays/{dev,prod}}
The file and folder structure looks like this:
── nginx-example
├── base
│ ├── base-configmap.yaml
│ ├── base-deployment.yaml
│ ├── base-service.yaml
│ └── kustomization.yaml
└── overlays
├── dev
│ ├── configmap-patch.yaml
│ └── kustomization.yaml
└── prod
├── configmap-patch.yaml
├── deployment-patch.yaml
└── kustomization.yaml
Base Manifests #
Base Deployment #
base/base-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine3.20
volumeMounts:
- name: nginx-html
mountPath: /usr/share/nginx/html
restartPolicy: Always
volumes:
- name: nginx-html
configMap:
name: nginx-html
Base ConfigMap #
base/base-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-html
data:
index.html: |
Nginx Example
Base Service #
base/base-service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
labels:
app: nginx
spec:
selector:
app: nginx
ports:
- port: 8080 # Port within the cluster
targetPort: 80 # Pod port
nodePort: 30080 # External port
type: NodePort
Base kustomization.yaml #
base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
metadata:
name: nginx-example
commonLabels:
app: nginx
resources:
- base-configmap.yaml
- base-deployment.yaml
- base-service.yaml
-
metadataThis section is used in Kubernetes resources to store data that helps uniquely identify the object. It usually includes fields like name, namespace, labels. -
commonLabelsThis section is used to define common labels that will be applied to all resources in this Kustomize configuration. -
resourcesLists the Kubernetes resources that should be managed by Kustomize. By default, Kustomize connects the resources in the order they appear and applies any overlays if they are specified.
Dev Overlay #
Overlay kustomization.yaml #
overlays/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: nginx-dev # Specify namespace for dev overlay
resources:
- ../../base
patches:
- path: configmap-patch.yaml
- target:
kind: Service
name: nginx-service
patch: |
- op: replace
path: /spec/ports/0/nodePort
value: 30081
- op: replace
path: /spec/ports/0/port
value: 8081
namespace: nginx-dev
commonLabels:
env: dev
Overlay ConfigMap Patch #
overlays/dev/configmap-patch.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-html
data:
index.html: |
Nginx Dev
Production Overlay #
Overlay kustomization.yaml #
overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
namespace: nginx-prod # Specify namespace for production overlay
patches:
- path: configmap-patch.yaml
- path: deployment-patch.yaml
- target:
kind: Service
name: nginx-service
patch: |
- op: replace
path: /spec/ports/0/nodePort
value: 30082
- op: replace
path: /spec/ports/0/port
value: 8082
commonLabels:
env: prod
Overlay ConfigMap Patch #
overlays/prod/configmap-patch.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-html
data:
index.html: |
Nginx Production
Overlay Deployment Patch #
- overlays/prod/deployment-patch.yaml`
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3 # More replicas for production
template:
metadata:
labels:
version: prod
Apply the Overlays #
Verify the Merged Configuration #
# Test the "prod" overlay configuration
kubectl kustomize overlays/prod/
# Shell output:
apiVersion: v1
data:
index.html: |
Nginx Production
kind: ConfigMap
metadata:
labels:
app: nginx
env: prod
name: nginx-html
namespace: nginx-prod
---
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
env: prod
name: nginx-service
namespace: nginx-prod
spec:
ports:
- nodePort: 30082
port: 8082
targetPort: 80
selector:
app: nginx
env: prod
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx
env: prod
name: nginx-deployment
namespace: nginx-prod
spec:
replicas: 3
selector:
matchLabels:
app: nginx
env: prod
template:
metadata:
labels:
app: nginx
env: prod
version: prod
spec:
containers:
- image: nginx:alpine3.20
name: nginx
volumeMounts:
- mountPath: /usr/share/nginx/html
name: nginx-html
restartPolicy: Always
volumes:
- configMap:
name: nginx-html
name: nginx-html
Deploy the Overlays #
# Apply the "dev" overlay
kubectl apply -k overlays/dev
# Apply the "prod" overlay
kubectl apply -k overlays/prod
Verify the Deployment #
Verify the “dev” deployment:
# List all resources in "nginx-dev" namespace
kubectl get all -n nginx-dev
# Shell output:
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-565dbd744d-4r77v 1/1 Running 0 4s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx-service NodePort 10.43.205.201 <none> 8081:30081/TCP 4s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 1/1 1 1 4s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deployment-565dbd744d 1 1 1 4s
Verify the “prod” deployment:
# List all resources in "nginx-prod" namespace
kubectl get all -n nginx-prod
# Shell output:
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-5fd55f9bc-k7lmq 1/1 Running 0 9s
pod/nginx-deployment-5fd55f9bc-lt8h6 1/1 Running 0 9s
pod/nginx-deployment-5fd55f9bc-mfl59 1/1 Running 0 9s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx-service NodePort 10.43.100.80 <none> 8082:30082/TCP 9s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 3/3 3 3 9s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deployment-5fd55f9bc 3 3 3 9s
Test the Overlay Deployments #
Test the “dev” deployment
# Curl the "dev" overlay
curl 192.168.30.11:30081
# Shell output:
Nginx Dev
Test the “prod” deployment
# Curl the "dev" overlay
curl 192.168.30.11:30082
# Shell output:
Nginx Production
Delete the Overlay Deployments #
# Delete the overlay deployments
kubectl delete -k overlays/dev
kubectl delete -k overlays/prod
Links #
# Documentation
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/
# Official Documentation
https://kustomize.io/