↓Skip to main content

Deploying with Flux: Deploy Manifests from public and private GitHub Repositories into Kubernetes Cluster, Flux Commands

1701 words·
Flux GitHub Kubernetes K8s
Table of Contents
Flux - This article is part of a series.
Part 1: This Article
Part 2: Deploying with Flux: Deploy Manifests from public and private GitLab.com Repositories into Kubernetes Cluster
Part 3: Deploying with Flux: Deploy Manifests from private Repositories of a self-hosted GitLab Instance into Kubernetes Cluster

Overview
#

I’m using a Kubeadm based K8s cluster with one controller and two worker nodes, with MetalLB and Nginx Ingress.

Make sure the Use Kubernetes Version 1.28, otherwise the latest Flux version (2.3.0) want work.

NAME      STATUS   ROLES           AGE     VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE           KERNEL-VERSION     CONTAINER-RUNTIME
ubuntu1   Ready    control-plane   9m38s   v1.28.11   192.168.30.10   <none>        Ubuntu 24.04 LTS   6.8.0-35-generic   containerd://1.7.18
ubuntu2   Ready    worker          7m19s   v1.28.11   192.168.30.11   <none>        Ubuntu 24.04 LTS   6.8.0-35-generic   containerd://1.7.18
ubuntu3   Ready    worker          7m15s   v1.28.11   192.168.30.12   <none>        Ubuntu 24.04 LTS   6.8.0-35-generic   containerd://1.7.18

GitHub
#

Create GitHub Access Token
# 

# token
ghp_q1R...

Public Deployment Project
#

Create Project
#

I’m using my public GitHub repository named “k8s-example-deployment” to deploy it’s manifests into the Kubernetes cluster.

Repository URL: https://github.com/jueklu/k8s-example-deployment


Deployment Manifest
#

Create the following deployment manifest:

# Add the following file to the "k8s-example-deployment" repository:
deployments/nginx-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: default
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

The folderstructure looks like this

├── deployments
│   └── nginx-deployment.yaml
└── README.md

Private Deployment Project
#

Create Project
#

I’m using my private GitHub repository named “k8s-example-deployment-private” to deploy it’s manifests into the Kubernetes cluster.

Repository URL: https://github.com/jueklu/k8s-example-deployment-private


Deployment Manifest
#

Create the following deployment manifest:

# Add the following file to the "k8s-example-deployment" repository:
deployments/nginx-deployment-private.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment-private
  namespace: default
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

The folderstructure looks like this

├── deployments
│   └── nginx-deployment-private.yaml
└── README.md

Kubernetes Cluster
#

Install Flux CLI
# 

# Install Flux CLI
curl -s https://fluxcd.io/install.sh | sudo bash

# Verify installation / check verison
flux --version

Verify Flux Compatibility
# 

# Check Kubernetes compatibility
flux check --pre

# Shell output:
► checking prerequisites
✔ Kubernetes 1.28.11 >=1.28.0-0
✔ prerequisites checks passed

Bootstrap Flux
# 

# Export GitHub credentials
export GITHUB_TOKEN=ghp_q1R...
export GITHUB_USER=jueklu

# Run the bootstrap command
flux bootstrap github \
  --owner=$GITHUB_USER \
  --repository=flux-repo \
  --branch=main \
  --path=./clusters/my-cluster \
  --personal

# Shell output
► connecting to github.com
✔ repository "https://github.com/jueklu/flux-repo" created
► cloning branch "main" from Git repository "https://github.com/jueklu/flux-repo.git"
✔ cloned repository
► generating component manifests
✔ generated component manifests
✔ committed component manifests to "main" ("a3e7a78f99b7287ed4e806d2b5f")
► pushing component manifests to "https://github.com/jueklu/flux-repo.git"
► installing components in "flux-system" namespace
✔ installed components
✔ reconciled components
► determining if source secret "flux-system/flux-system" exists
► generating source secret
✔ public key: ecdsa-sha2-...Fq+O6vSr2iz2/xyOMSpi0A==
✔ configured deploy key "flux-system-main-flux-system-./clusters/my-cluster" for "https://github.com/jueklu/flux-repo"
► applying source secret "flux-system/flux-system"
✔ reconciled source secret
► generating sync manifests
✔ generated sync manifests
✔ committed sync manifests to "main" ("daf0fa074bd4211fc142b5cde5cf87bd4b")
► pushing sync manifests to "https://github.com/jueklu/flux-repo.git"
► applying sync manifests
✔ reconciled sync configuration
◎ waiting for GitRepository "flux-system/flux-system" to be reconciled
✔ GitRepository reconciled successfully
◎ waiting for Kustomization "flux-system/flux-system" to be reconciled
✔ Kustomization reconciled successfully
► confirming components are healthy
✔ helm-controller: deployment ready
✔ kustomize-controller: deployment ready
✔ notification-controller: deployment ready
✔ source-controller: deployment ready
✔ all components are healthy

Verify Flux Pods
#

Verify the Flux pods are up and running:

# List pods in "flux-system" namespace
kubectl get pods -n flux-system

# Shell output:
NAME                                       READY   STATUS    RESTARTS   AGE
helm-controller-5f7457c9dd-sw7l9           1/1     Running   0          115s
kustomize-controller-5f58d55f76-b7zdm      1/1     Running   0          115s
notification-controller-685bdc466d-wqq2h   1/1     Running   0          115s
source-controller-86b8b57796-mwjnb         1/1     Running   0          115s

Check Flux Logs
# 

# Check the Flux logs
flux logs

# Shell output:
2024-07-06T06:27:48.212Z info Kustomization/flux-system.flux-system - Source artifact not found, retrying in 30s
2024-07-06T06:27:55.123Z info Kustomization/flux-system.flux-system - server-side apply for cluster definitions completed
2024-07-06T06:27:55.337Z info Kustomization/flux-system.flux-system - server-side apply completed
2024-07-06T06:27:55.350Z info Kustomization/flux-system.flux-system - Reconciliation finished in 890.602069ms, next run in 10m0s
2024-07-06T06:28:18.618Z info Kustomization/flux-system.flux-system - server-side apply for cluster definitions completed
2024-07-06T06:28:18.729Z info Kustomization/flux-system.flux-system - server-side apply completed
2024-07-06T06:28:18.744Z info Kustomization/flux-system.flux-system - Reconciliation finished in 525.194959ms, next run in 10m0s
2024-07-06T06:27:54.452Z info GitRepository/flux-system.flux-system - stored artifact for commit 'Add Flux sync manifests'
2024-07-06T06:28:52.581Z info GitRepository/flux-system.flux-system - no changes since last reconcilation: observed revision 'main@sha1:daf0fa074bd4211fc1800c9242b5cde5cf87bd4b'

Flux Repository
#

Clone Repository
#

Clone the GitHub repository “flux-repo” created by Flux to a local host:

# Clone the Flux repository
git clone git@github.com:jueklu/flux-repo.git

The folder structure should look like this:

flux-repo
└── clusters
    └── my-cluster
        └── flux-system
            ├── gotk-components.yaml
            ├── gotk-sync.yaml
            └── kustomization.yaml

Install Flux CLI
#

Install the Flux CLI on the host where the Flux repository was cloned:

# Install Flux CLI
curl -s https://fluxcd.io/install.sh | sudo bash

Public Deployment Project
#

Add Repository Source
#

Define the GitHub example project “k8s-example-deployment” as source:

# Create repository source
flux create source git k8s-example-deployment \
  --url=https://github.com/jueklu/k8s-example-deployment \
  --branch=main \
  --interval=1m \
  --export > clusters/my-cluster/k8s-example-deployment-source.yaml

Add Repository Kustomization
#

This will configure Flux to build and apply the manifests in the “deployments” directory:

# Create a kustomization
flux create kustomization k8s-example-deployment \
  --target-namespace=default \
  --source=k8s-example-deployment \
  --path="./deployments" \
  --prune=true \
  --wait=true \
  --interval=30m \
  --retry-interval=2m \
  --health-check-timeout=3m \
  --export > ./clusters/my-cluster/k8s-example-deployment-kustomization.yaml

Verify Folder Structure
#

The folder structure should now look like this

# Flux folder structure
clusters
└── my-cluster
    ├── flux-system
    │   ├── gotk-components.yaml
    │   ├── gotk-sync.yaml
    │   └── kustomization.yaml
    ├── k8s-example-deployment-kustomization.yaml
    └── k8s-example-deployment-source.yaml

Push
#

Push the kustomization and source manifests into the Flux GitHub repository:

# configure git
git config --global user.email "juergen@jklug.work"
git config --global user.name "Juergen"

# Add die manifests and commit
git add -A && git commit -m "Add example Source and Kustomization"

# Push changes to GitHub
git push

Watch Flux Deployment
# 

# Watch the Flux deployment
flux get kustomizations --watch

# Shell output:
k8s-example-deployment          False   Unknown Reconciliation in progress
k8s-example-deployment          False   True    Applied revision: main@sha1:3294f738
...

Verify the Deployment
# 

# List pods in the default namespace
kubectl get pods

# Shell output:
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-7c79c4bf97-878v8   1/1     Running   0          3m1s
nginx-deployment-7c79c4bf97-bbcsb   1/1     Running   0          3m1s
nginx-deployment-7c79c4bf97-flvbt   1/1     Running   0          3m1s

Private Deployment Project
#

Create SSH Key for Flux
#

On the Kubernetes Controller node create a SSH key pair for Flux:

# Create a Deploy Key for GitHub
ssh-keygen -t rsa -b 4096 -f flux-deploy-key

# Copy the public SSH key
cat flux-deploy-key.pub

Add SSH Key to GitHub Repository
#

Add the SSH public key to the GitHub repository “k8s-example-deployment-private”.

  • Open the GitHub repository

  • Go to “Settings” > (Security) “Deploy keys”

  • Click “Add deploy key”

  • Add a title for the key like Flux-Key

  • Paste the public SSH key

# Public SSH key
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDbUj+sQzeM+c99OzBSWad4v9Qxvy8ZS6ohri092axWKK5jfWDrttJsUL89Ohc6YWW5w1UV6p/BU4R7S1osZWxIdW3MtF5L4GJo6+r0c9whqddcfrXB3iJxruWRb7OggCDKtIj4p1eAzGqHc8w83RmWt/DocXfLP8Ykdwq5q...

Kubernetes Secret
#

Create a Kubernetes secret with the Flux SSH private key. The following command also adds the host key for the GitHub repository:

# Create Kubernetes secret (Run command in same dir as the SSH key)
flux create secret git flux-git-deploy \
  --url=ssh://git@github.com/jueklu/k8s-example-deployment-private \
  --private-key-file=./flux-deploy-key \
  --namespace=flux-system

# Delete Kubernetes secret
kubectl delete secret flux-git-deploy -n flux-system

# Optional: Verify / decode the Kubernetes secret
kubectl get secret flux-git-deploy -n flux-system -o jsonpath="{.data.identity}" | base64 --decode

Add Repository Source
#

Define the GitHub example project “k8s-example-deployment” as source:

# Create repository source
flux create source git k8s-example-deployment-private \
  --url=ssh://git@github.com/jueklu/k8s-example-deployment-private \
  --branch=main \
  --interval=1m \
  --secret-ref=flux-git-deploy \
  --export > clusters/my-cluster/k8s-example-deployment-private-source.yaml

Add Repository Kustomization
#

This will configure Flux to build and apply the manifests in the “deployments” directory:

# Create a kustomization
flux create kustomization k8s-example-deployment-private \
  --target-namespace=default \
  --source=k8s-example-deployment-private \
  --path="./deployments" \
  --prune=true \
  --wait=true \
  --interval=30m \
  --retry-interval=2m \
  --health-check-timeout=3m \
  --export > ./clusters/my-cluster/k8s-example-deployment-private-kustomization.yaml

Verify Folder Structure
#

The folder structure should now look like this

# Flux folder structure
 clusters
  └── my-cluster
    ├── flux-system
    │   ├── gotk-components.yaml
    │   ├── gotk-sync.yaml
    │   └── kustomization.yaml
    ├── k8s-example-deployment-private-kustomization.yaml
    └── k8s-example-deployment-private-source.yaml

Push
#

Push the kustomization and source manifests into the Flux GitHub repository:

# configure git
git config --global user.email "juergen@jklug.work"
git config --global user.name "Juergen"

Note: Make sure to push the source manifest first and then the kustomization manifest.

# Add the manifests and commit
git add -A && git commit -m "Add example Source and Kustomization"

# Push changes to GitHub
git push

Verify the Deployment
# 

# List pods in the default namespace
kubectl get pods

# Shell output:
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-deployment-private-7c79c4bf97-hmpjk   1/1     Running   0          14m
nginx-deployment-private-7c79c4bf97-vdmfh   1/1     Running   0          14m
nginx-deployment-private-7c79c4bf97-vdzxm   1/1     Running   0          14m

Flux Commands
#

Flux Status
# 

# Check Flux status
flux check

# Shell output:
► checking prerequisites
✔ Kubernetes 1.28.11 >=1.28.0-0
► checking version in cluster
✔ distribution: flux-v2.3.0
✔ bootstrapped: true
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v1.0.1
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.3.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.3.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.3.0
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta3
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1
✔ helmreleases.helm.toolkit.fluxcd.io/v2
✔ helmrepositories.source.toolkit.fluxcd.io/v1
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta3
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed

Restart Flux
# 

# Restart Flux
kubectl -n flux-system rollout restart deployment

Update Flux Manifests
# 

# Triggers Flux to pull the latest changes from the flux repository
flux reconcile source git flux-system

Sources & Kustomizations
#

List, Reconcile & Delete
# 

# List Kustomizations
flux get kustomizations

# Reconcile Kustomization: Synchronize with the latest state from the repository
flux reconcile kustomization k8s-example-deployment

# Watch Kustomizations
flux get kustomizations --watch

# Delete Kustomization
flux delete kustomization k8s-example-deployment

# List sources
flux get sources git

# Reconcile a Source: Synchronize with the latest state from the repository
flux reconcile source git k8s-example-deployment

# Delete Source
flux delete source git k8s-example-deployment

# List all Sources & Kustomizations
flux get all

# Shell output:
NAME                                    REVISION                SUSPENDED       READY   MESSAGE
gitrepository/flux-system               main@sha1:46dccb60      False           True    stored artifact for revision 'main@sha1:46dccb60'
gitrepository/k8s-example-deployment    main@sha1:3294f738      False           True    stored artifact for revision 'main@sha1:3294f738'

NAME                                    REVISION                SUSPENDED       READY   MESSAGE
kustomization/flux-system               main@sha1:46dccb60      False           True    Applied revision: main@sha1:46dccb60
kustomization/k8s-example-deployment    main@sha1:3294f738      False           True    Applied revision: main@sha1:3294f738

Suspend and Resume
# 

# Suspend Kustomization: Temporarily stops the reconciliation of a kustomization without affecting the current state in the cluster
flux suspend kustomization k8s-example-deployment

# Verify suspension
flux get kustomizations

# Shell output:
NAME                    REVISION                SUSPENDED       READY   MESSAGE
flux-system             main@sha1:46dccb60      False           True    Applied revision: main@sha1:46dccb60
k8s-example-deployment  main@sha1:3294f738      True            True    Applied revision: main@sha1:3294f738

# Resume Kustomization
flux resume kustomization k8s-example-deployment

Links # 

# Flux SSH Kubernetes Secret
https://fluxcd.io/flux/components/source/gitrepositories/
Flux - This article is part of a series.
Part 1: This Article
Part 2: Deploying with Flux: Deploy Manifests from public and private GitLab.com Repositories into Kubernetes Cluster
Part 3: Deploying with Flux: Deploy Manifests from private Repositories of a self-hosted GitLab Instance into Kubernetes Cluster