↓Skip to main content

Argo CD ApplicationSets with GitLab, Harbor and Ansible

2215 words·
Argo CD GitLab Harbor Ansible ApplicationSet
Table of Contents

Prerequisites
#

SSH Key Pairs
#

Helm Repo Pipeline Key
#

Create a private SSH key pair, that is used by the GitLab code repostory pipeline to access the GitLab Helm Chart repository:

# Create SSH key pair
ssh-keygen -t rsa -b 2048 -C "helm_repo_key" -f  ~/.ssh/helm_repo_key

  • helm_repo_key Private SSH key, added as encoded CI/CD variable to the GitLab code repository

  • helm_repo_key.pub Public SSH key, add to GitLab Helm repository as deploy key / pipeline key


# encode the pipeline key: Test version
base64 -w0 ~/.ssh/helm_repo_key > helm_repo_key.b64

# List encoded key
cat helm_repo_key.b64

# Shell output:
LS0tLS1CRUdJTiBPUEVOU1NIIFBSSVZBVEUgS0VZLS0tLS0KYjNCbGJuTnphQzFyWlhrdGRqRUFBQUFBQkc1dmJtVUFBQUFFYm05dVpRQUFBQUFBQUFBQkFBQUJGd0FBQUFkemMyZ3RjbgpOaEFBQUFBd0VBQVFBQUFRRUF3TElHOFVRVU8wYzdXZFp0d3B2OFlycDJWaU1KZVhDbnpXWW5iamh4SjNIa0lUd09QalFMCk5NaGx0T29lQVdNMEt6cGErdU5IQWl3V1VGWUl4TzdFOVo0NjJqTjRSTU9BWHRlSU1sZjJRekJwTHZsK1QyUnpkelRnQ0sKU3NING9IMjFmYlFBd28rdlg0aXE1UWROK1BHY2kzZ2hGcGRISE9rZXdpN1FwbDdxRjdlcHVDRkVoRXVQcDlyTXNHQVJlSQp6eGxtV1JKMmF4YTh4aTIrbktmRTdzMWlmQnQvQzFpZDhiVElUY3NlVGJIcmx4RkcvcEJTblRyUmZMUVNKemw5L2tqQ2hoCmY2OEE2V1I2Mi8xaXNZdHZvR1lvV1JrT2Nqejc1VVJDQkl4NXF2UWJOVHV2dEwrZjV4YkJ3bTZ6Ty9pSHZPVzV0dElhVm4KdVVHelJJUm9jUUFBQThqRlluZWV4V0ozbmdBQUFBZHpjMmd0Y25OaEFBQUJBUURBc2dieFJCUTdSenRaMW0zQ20veGl1bgpaV0l3bDVjS2ZOWmlkdU9IRW5jZVFoUEE0K05BczB5R1cwNmg0Qll6UXJPbHI2NDBjQ0xCWlFWZ2pFN3NUMW5qcmFNM2hFCnc0QmUxNGd5Vi9aRE1Ha3UrWDVQWkhOM05PQUlwS3dmaWdmYlY5dEFEQ2o2OWZpS3JsQjAzNDhaeUxlQ0VXbDBjYzZSN0MKTHRDbVh1b1h0Nm00SVVTRVM0K24yc3l3WUJGNGpQR1daWkVuWnJGcnpHTGI2Y3A4VHV6V0o4RzM4TFdKM3h0TWhOeXg1TgpzZXVYRVViK2tGS2RPdEY4dEJJbk9YMytTTUtHRi9yd0RwWkhyYi9XS3hpMitnWmloWkdRNXlQUHZsUkVJRWpIbXE5QnMxCk82KzB2NS9uRnNIQ2JyTTcrSWU4NWJtMjBocFdlNVFiTkVoR2h4QUFBQUF3RUFBUUFBQVFBUGdUT2NUYU5JR3NUc2gxdXAKRFVhTFJpQVg4YlZGTTNxWkJSZDJuNjJCSkVHY0ovb29hS1VmTCtHNUl4NWJjN2xDb2kwb1kwbTNJOGQrMnRGVlVUWUJZbQpRdll2VlNLUjNtNDdaR2dMdmtyNXhEQnBteWR5MXdRYzJsL0dRS0hKL2h0d25COTBpRDJ1OU83MVF6ZHpxTE0xS0tzVzU0CkMyRTQ3aFJvV000WFczeXkxSjJJYjhDZHdaNlZibHdjbWRRZWM4TE5QOXhqK1JhTWNTbXhuV3oxS1F1dFovY2NtWGk0ckoKWEhIdXpKeERJZVhrR0ZZcFZZWmhMcVZYRXFkdHdqbEM1YVNodjhDajdwazBreTRpWHlkVytWL2krdEh5SVpuc29ocG5kRgpVVXZGRXdtcFZSbnpCY0l4ZEtWV09sU1FZcENiNnR4dm92Zko0aWVoT0hTTkFBQUFnUURDbHlEd21qUDc2aWRGcHdsdUdxCncyaXVxcTlFRlJsVHRhckhsV01PUXRlN3BCUGxvMG9yQnFvOGVQYjlxbFhMWG1rMlRTZnhrUjQ4SWJsZTlLVy9OWmluVnEKd0tOU0NpZHozUlRXZ0xFdnU0c1dCNXE0ZmVjUVZGeW81SGlZM0FmQ3VldjF4ZEhtdHArSWU1dXpta3hMTXduc0tBam12ego0cXFUR3B6M1J4TXdBQUFJRUE1UktKVnovSkhQQkRoeCtMVVcvcStYcXJHQ2RmSlZDeng1VlhqQTZBWXhaNGM5Qlh3UDA0CkVzMnFXUXJyVFpXRGpFUEJDYmNTOStNR2h5Q0dlNmVTYjl6Rzl0eUFKYXdiVVgxZ25nK0xOcWVIY04vMWwzVzYyajNGbGEKTGdEczhhWDNCTzZyUU1nNTQvZkwrK2tuZ01VMm1GNUNPdlFMRnZQamdwZWdaQ1R1MEFBQUNCQU5kWXpNT1VsVDNmV2JNNgpGQjdLSlpJU094bXp5VURTMm8xU3ZJaXAvVnlBTHJ2WVBTSVNJTkc0amg3VlVYNEk1ejZkSmJHQmY0Y2YxMVVJWVlKQ1ZpCml0U0VWU2pPSnpaK1NSR25qWlF3aDkvcDJNU0drZU5FeUxKTityUWtza2ZsMmNqdW5zNlRYWmxPRWZiZlpnQkxXUytDWDcKQlp0dHh0a0NNaHhRdHNzVkFBQUFEV2hsYkcxZmNtVndiMTlyWlhrQkFnTUVCUT09Ci0tLS0tRU5EIE9QRU5TU0ggUFJJVkFURSBLRVktLS0tLQo=

Argo CD Repository Key
#

Create a private SSH key pair, that is used by Argo CD to access the GitLab Helm Chart repository:

# Create SSH key pair
ssh-keygen -t rsa -b 2048 -C "deploy_key" -f ~/.ssh/deploy_key

  • deploy_key Private SSH key, used by Argo CD

  • deploy_key.pub Public SSH key, add to GitLab Helm repository as deploy key


Harbor Project and Credentials
#

I’m using the following Harbor project and credentials:

Harbor Project name: argocd-applicationset-1

User: robot$argocd-applicationset-1+applicationset-admin

Password: 1VH306D0DUjDLO89sHKt2hnCuoj3MPgq



GitLab Code Repository: Main (Dev)
#

File and Folder Structure
# 

# applicationset-1-code
├── Dockerfiles
│   └── Dockerfile
├── .gitlab-build.yml
├── .gitlab-ci.yml
├── .gitlab-dev.yml
├── .gitlab-prod.yml
├── .gitlab-update-helm.yml
├── README.md
└── website
    └── index.html

CI/CD Variables
#

Harbor Credentials
#

Add the credentials for the Harbor Project as GitLab CI/CD Variables:

# Key
HARBOR_USERNAME

# Value
robot$argocd-applicationset-1+applicationset-admin

# Key
HARBOR_PASSWORD

# Value
1VH306D0DUjDLO89sHKt2hnCuoj3MPgq

Variable options:

  • Enable “Visibility” x “Masked”

  • Disable Flag “Protect variable”


Encoded SSH Key
#

Add the encoded private SSH (Helm Repo Pipeline Key) as GitLab CI/CD Variable:

# Key
HELM_REPO_KEY

# Value
LS0tLS1CRUdJTiBPUEVOU1NIIFBSSVZBVEUgS0VZLS0tLS0KYjNCbGJuTnphQzFyWlhrdGRqRUFBQUFBQkc1dmJtVUFBQUFFYm05dVpRQUFBQUFBQUFBQkFBQUJGd0FBQUFkemMyZ3RjbgpOaEFBQUFBd0VBQVFBQUFRRUF3TElHOFVRVU8wYzdXZFp0d3B2OFlycDJWaU1KZVhDbnpXWW5iamh4SjNIa0lUd09QalFMCk5NaGx0T29lQVdNMEt6cGErdU5IQWl3V1VGWUl4TzdFOVo0NjJqTjRSTU9BWHRlSU1sZjJRekJwTHZsK1QyUnpkelRnQ0sKU3NING9IMjFmYlFBd28rdlg0aXE1UWROK1BHY2kzZ2hGcGRISE9rZXdpN1FwbDdxRjdlcHVDRkVoRXVQcDlyTXNHQVJlSQp6eGxtV1JKMmF4YTh4aTIrbktmRTdzMWlmQnQvQzFpZDhiVElUY3NlVGJIcmx4RkcvcEJTblRyUmZMUVNKemw5L2tqQ2hoCmY2OEE2V1I2Mi8xaXNZdHZvR1lvV1JrT2Nqejc1VVJDQkl4NXF2UWJOVHV2dEwrZjV4YkJ3bTZ6Ty9pSHZPVzV0dElhVm4KdVVHelJJUm9jUUFBQThqRlluZWV4V0ozbmdBQUFBZHpjMmd0Y25OaEFBQUJBUURBc2dieFJCUTdSenRaMW0zQ20veGl1bgpaV0l3bDVjS2ZOWmlkdU9IRW5jZVFoUEE0K05BczB5R1cwNmg0Qll6UXJPbHI2NDBjQ0xCWlFWZ2pFN3NUMW5qcmFNM2hFCnc0QmUxNGd5Vi9aRE1Ha3UrWDVQWkhOM05PQUlwS3dmaWdmYlY5dEFEQ2o2OWZpS3JsQjAzNDhaeUxlQ0VXbDBjYzZSN0MKTHRDbVh1b1h0Nm00SVVTRVM0K24yc3l3WUJGNGpQR1daWkVuWnJGcnpHTGI2Y3A4VHV6V0o4RzM4TFdKM3h0TWhOeXg1TgpzZXVYRVViK2tGS2RPdEY4dEJJbk9YMytTTUtHRi9yd0RwWkhyYi9XS3hpMitnWmloWkdRNXlQUHZsUkVJRWpIbXE5QnMxCk82KzB2NS9uRnNIQ2JyTTcrSWU4NWJtMjBocFdlNVFiTkVoR2h4QUFBQUF3RUFBUUFBQVFBUGdUT2NUYU5JR3NUc2gxdXAKRFVhTFJpQVg4YlZGTTNxWkJSZDJuNjJCSkVHY0ovb29hS1VmTCtHNUl4NWJjN2xDb2kwb1kwbTNJOGQrMnRGVlVUWUJZbQpRdll2VlNLUjNtNDdaR2dMdmtyNXhEQnBteWR5MXdRYzJsL0dRS0hKL2h0d25COTBpRDJ1OU83MVF6ZHpxTE0xS0tzVzU0CkMyRTQ3aFJvV000WFczeXkxSjJJYjhDZHdaNlZibHdjbWRRZWM4TE5QOXhqK1JhTWNTbXhuV3oxS1F1dFovY2NtWGk0ckoKWEhIdXpKeERJZVhrR0ZZcFZZWmhMcVZYRXFkdHdqbEM1YVNodjhDajdwazBreTRpWHlkVytWL2krdEh5SVpuc29ocG5kRgpVVXZGRXdtcFZSbnpCY0l4ZEtWV09sU1FZcENiNnR4dm92Zko0aWVoT0hTTkFBQUFnUURDbHlEd21qUDc2aWRGcHdsdUdxCncyaXVxcTlFRlJsVHRhckhsV01PUXRlN3BCUGxvMG9yQnFvOGVQYjlxbFhMWG1rMlRTZnhrUjQ4SWJsZTlLVy9OWmluVnEKd0tOU0NpZHozUlRXZ0xFdnU0c1dCNXE0ZmVjUVZGeW81SGlZM0FmQ3VldjF4ZEhtdHArSWU1dXpta3hMTXduc0tBam12ego0cXFUR3B6M1J4TXdBQUFJRUE1UktKVnovSkhQQkRoeCtMVVcvcStYcXJHQ2RmSlZDeng1VlhqQTZBWXhaNGM5Qlh3UDA0CkVzMnFXUXJyVFpXRGpFUEJDYmNTOStNR2h5Q0dlNmVTYjl6Rzl0eUFKYXdiVVgxZ25nK0xOcWVIY04vMWwzVzYyajNGbGEKTGdEczhhWDNCTzZyUU1nNTQvZkwrK2tuZ01VMm1GNUNPdlFMRnZQamdwZWdaQ1R1MEFBQUNCQU5kWXpNT1VsVDNmV2JNNgpGQjdLSlpJU094bXp5VURTMm8xU3ZJaXAvVnlBTHJ2WVBTSVNJTkc0amg3VlVYNEk1ejZkSmJHQmY0Y2YxMVVJWVlKQ1ZpCml0U0VWU2pPSnpaK1NSR25qWlF3aDkvcDJNU0drZU5FeUxKTityUWtza2ZsMmNqdW5zNlRYWmxPRWZiZlpnQkxXUytDWDcKQlp0dHh0a0NNaHhRdHNzVkFBQUFEV2hsYkcxZmNtVndiMTlyWlhrQkFnTUVCUT09Ci0tLS0tRU5EIE9QRU5TU0ggUFJJVkFURSBLRVktLS0tLQo=

Variable options:

  • Enable “Visibility” x “Masked”

  • Disable Flag “Protect variable”


Verify Variables
#



Pipeline Manifests
#

.gitlab-ci.yml
#

  • .gitlab-ci.yml
variables:
  GITLAB_URL: "gitlab.jklug.work"
  HELM_REPOSITORY: "git@gitlab.jklug.work:production/argocd-applicationsets/applicationset-1-helm.git"  # Helm Chart Repository
  HARBOR_CONTAINER_REGISTRY: "harbor.jklug.work"
  HARBOR_PROJECT: "argocd-applicationset-1"
  HARBOR_REPOSITORY: "${HARBOR_CONTAINER_REGISTRY}/${HARBOR_PROJECT}/${DEPLOYMENT_ENVIRONMENT}/example-app"

default:
  interruptible: true

stages:
  - build
  - update-helm

workflow:
  name: '${PIPELINE_NAME} -Commit: ${CI_COMMIT_TITLE}'
  rules:
    - if: $CI_COMMIT_REF_NAME == "main"
      variables:
        PIPELINE_NAME: 'Main branch build'
        DEPLOYMENT_ENVIRONMENT: dev
        IMAGE_VERSION: "${CI_COMMIT_REF_SLUG}_${CI_PIPELINE_IID}"
        HELM_CHART_VERSION: $CI_PIPELINE_IID
        HELM_REPO_BRANCH: "main"

    - if: $CI_COMMIT_TAG
      variables:
        PIPELINE_NAME: "Tag $CI_COMMIT_TAG build"
        DEPLOYMENT_ENVIRONMENT: prod
        IMAGE_VERSION: "$CI_COMMIT_TAG"
        HELM_CHART_VERSION: $CI_COMMIT_TAG
        HELM_REPO_BRANCH: "prod"

include:
  - local: .gitlab-dev.yml
  - local: .gitlab-prod.yml
  - local: .gitlab-build.yml
  - local: .gitlab-update-helm.yml

.gitlab-build.yml
#

  • .gitlab-build.yml
.build:
  stage: build
  image: docker:28.4
  services:
    - name: docker:28.4-dind
      alias: docker
      command: ["--tls=false"]
  variables:
    DOCKER_TLS_CERTDIR: ""
  before_script:
    # Login to Harbor registry via robot user
    - docker login -u "$HARBOR_USERNAME" -p "$HARBOR_PASSWORD" "$HARBOR_CONTAINER_REGISTRY"
  script:
    # Define image tag name
    - IMAGE_TAG_COMPLETE="${HARBOR_REPOSITORY}:${IMAGE_VERSION}"
    # Build and tag image
    - docker build -f Dockerfiles/Dockerfile --pull -t "${IMAGE_TAG_COMPLETE}" .
    # Push image to Harbor
    - docker push "${IMAGE_TAG_COMPLETE}"

.gitlab-update-helm.yml
#

  • gitlab-update-helm.yml
.update-helm:
  stage: update-helm
  image: debian:12.12-slim
  script:
    # Install required tools
    - apt-get update
    - apt-get install -y --no-install-recommends git openssh-client ca-certificates curl
    - curl -L "https://github.com/mikefarah/yq/releases/download/v4.47.2/yq_linux_amd64" -o /usr/local/bin/yq
    - chmod +x /usr/local/bin/yq
    # Configure SSH for Git
    - mkdir -p ~/.ssh
    - echo "$HELM_REPO_KEY" | base64 -d > ~/.ssh/id_rsa
    - chmod 600 ~/.ssh/id_rsa
    - ssh-keyscan -H ${GITLAB_URL} >> ~/.ssh/known_hosts
    # Clone the external repository into helm-chart-repo directory
    - git clone --branch $HELM_REPO_BRANCH $HELM_REPOSITORY helm-chart-repo
    - cd helm-chart-repo
    # Verify the Helmchart does exist
    - ls -R
    - ls -R helm-chart
    # Update values.yaml with new container images
    - yq e ".example_app.repository = strenv(HARBOR_REPOSITORY) | .example_app.tag = strenv(IMAGE_VERSION)" -i helm-chart/values.yaml
    # Update Chart.yaml with new version
    - yq e ".version = \"$HELM_CHART_VERSION\" | .appVersion = \"$HELM_CHART_VERSION\"" -i helm-chart/Chart.yaml
    # Update Commit Message
    - yq e ".message = \"$CI_COMMIT_MESSAGE\"" -i helm-chart/Chart.yaml
    # Add and commit the changes
    - git config --global user.email "juergen@jklug.work"
    - git config --global user.name "GitLab CI"
    - git add helm-chart/values.yaml # Add values.yaml
    - git add helm-chart/Chart.yaml # Add Chart.yaml
    - git commit -m "Update Helm chart with image $VERSION"
    # Push the changes back to the external repository
    - git push origin $HELM_REPO_BRANCH

.gitlab-dev.yml
#

  • .gitlab-dev.yml
build-dev:
  extends:
    - .build
  rules:
    - if: $CI_COMMIT_REF_NAME == "main"

update-helm-dev:
  extends:
    - .update-helm
  rules:
    - if: $CI_COMMIT_REF_NAME == "main"
  needs:
    - job: build-dev  # Require build job

.gitlab-prod.yml
#

  • .gitlab-prod.yml
build-prod:
  extends:
    - .build
  rules:
    - if: $CI_COMMIT_TAG


update-helm-prod:
  extends:
    - .update-helm
  rules:
    - if: $CI_COMMIT_TAG
  needs:
    - job: build-prod  # Require build job



Dockerfile
#

  • Dockerfiles/Dockerfile
FROM nginx:1.18

COPY website/index.html /usr/share/nginx/html

Example Website
#

  • website/index.html
<!DOCTYPE html>
<html>
<head>
        <title>K8s ApplicationSet Deployment</title>
</head>
<body>
  <h1>Argo CD ApplicationSet</h1>
  <p>Example Deployment</p>
</body>
</html>



GitLab Code Repository: Prod
#

Create Prod Branch
# 

# Create a new "prod" branch and switch to it
git checkout -b prod

# Push local branch and establish tracking relationship between local branch and it's remote counterpart
git push -u origin prod

Create and Push Tag
#

The tag push triggers the GitLab pipeline for the prod environment:

# Create local tag
git tag 1.0.1

# Push tag to remote repository
git push origin 1.0.1



GitLab Helm Repository
#

File and Folder Structure
# 

# applicationset-1-helm
├── helm-chart
│   ├── Chart.yaml
│   ├── templates
│   │   ├── deployment.yaml
│   │   ├── ingress.yaml
│   │   └── service.yaml
│   └── values.yaml
└── README.md

# Create file and folder structure
mkdir -p helm-chart/templates &&
touch helm-chart/{Chart.yaml,values.yaml} &&
touch helm-chart/templates/deployment.yaml &&
touch helm-chart/templates/service.yaml &&
touch helm-chart/templates/ingress.yaml

Deploy Keys
#

Add Pipeline Key
#

Add the previously created public SSH key, so that GitLab code repository pipeline can access the Helm chart repository:

  • Go to: (Settings) > “Repository” > “Deploy keys”

  • Click “Add new key”

  • Define a title like “helm_repo_key”

  • Paste the public SSH key helm_repo_key.pub into the “Key” section

  • Enable “Grant write permissions to this key”

  • Click “Add key”


Add Deploy Key
#

Add the previously created public SSH key, so that Argo CD can access the Helm chart repository:

  • Go to: (Settings) > “Repository” > “Deploy keys”

  • Click “Add new key”

  • Define a title like “argocd_deploy”

  • Paste the public SSH key into the “Key” section

  • Disable “Grant write permissions to this key”

  • Click “Add key”

Verify keys
#



Helm Chart: Dev Branch
#

values.yaml
#

  • helm-chart/values.yaml
# Define the image details
example_app:
  repository: ""  # Registry will be updated by the pipeline
  tag: ""  # Tag will be updated by the pipeline
  container_target_port: 80
  ingress_domain: example-app-dev.jklug.work
imagePullSecrets:
  - harbor-registry-secret # The Kubernetes secret name

Chart.yaml
# 

apiVersion: v2
name: Example App
description: Helm chart for Example App
type: application
version: ""
appVersion: ""
message: |

deployment.yaml
#

  • helm-chart/templates/deployment.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  labels:
    app: example-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example-app
  template:
    metadata:
      labels:
        app: example-app
    spec:
      imagePullSecrets:
      - name: "{{ .Values.imagePullSecrets | first }}"
      containers:
      - name: example-app
        image: "{{ .Values.example_app.repository }}:{{ .Values.example_app.tag }}"
        ports:
        - containerPort: {{ .Values.example_app.container_target_port | int }}
        imagePullPolicy: IfNotPresent

service.yaml
#

  • helm-chart/templates/service.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: example-app-service
spec:
  type: ClusterIP
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 900
  selector:
    app: example-app
  ports:
    - name: http
      protocol: TCP
      port: {{ .Values.example_app.container_target_port | int }}
      targetPort: {{ .Values.example_app.container_target_port | int }}

ingress.yaml
#

  • helm-chart/templates/ingress.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-app-ingress
  annotations:
    #cert-manager.io/cluster-issuer: cluster-issuer-dns01
    nginx.ingress.kubernetes.io/ssl-redirect: "false"  # Set to true in prod
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
spec:
  ingressClassName: nginx-metallb
  #tls:
    #- hosts:
        #- "{{ .Values.example_app.ingress_domain }}"
      #secretName: "example-app-tls-secret"
  rules:
    - host: "{{ .Values.example_app.ingress_domain }}"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: example-app-service
                port:
                  number: {{ .Values.example_app.container_target_port | int }}

Helm Chart: Prod Branch
#

Create Branch
# 

# Create a new "prod" branch and switch to it
git switch --create prod

values.yaml
#

Adapt the Kubernetes Ingress URL of the values.yaml file:

  • helm-chart/values.yaml
# Define the image details
example_app:
  repository: ""  # Registry will be updated by the pipeline
  tag: ""  # Tag will be updated by the pipeline
  container_target_port: 80
  ingress_domain: example-app-prod.jklug.work
imagePullSecrets:
  - harbor-registry-secret # The Kubernetes secret name

Push Branch
# 

# Push local branch and establish tracking relationship between local branch and it's remote counterpart
git push -u origin prod



Ansible Playbook
#

Apply Argo CD ApplicationSet
# 

---
# Deploy Argo CD Example ApplicationSet
- name: Argo CD ApplicationSet
  hosts: localhost
  connection: local
  gather_facts: false
  vars:
    # Vault Secrets
    argocd_repository_keys: "{{ lookup('community.hashi_vault.vault_kv2_get', 'services/argocd/gitlab_deploykey', engine_mount_point='homelab_prod').secret }}"
    # Vault Variables
    argocd_repository_private_key: "{{ argocd_repository_keys.private_key }}"  # Key to access GitLab Helm chart repository
    # GitLab Helm Chart Repository
    gitlab_helm_repository: "git@gitlab.jklug.work:production/argocd-applicationsets/applicationset-1-helm.git"  # Define GitLab Helm chart repository
    gitlab_repository_name: "applicationset-1-helm"  # Just a unique name for the GitLab repository in ArgoCD
    # Harbor Credentials
    harbor_registry: "harbor.jklug.work"
    harbor_user: "robot$argocd-applicationset-1+applicationset-admin"  # Harbor credentials for image pull
    harbor_pw: "1VH306D0DUjDLO89sHKt2hnCuoj3MPgq"  # Harbor credentials for image pull
    # Deployment Variables
    app_namespace_dev: example-app-dev  # Define Kubernetes namespace
    app_namespace_prod: example-app-prod  # Define Kubernetes namespace
    # ArgoCD Variables
    argocd_namespace: "argocd"  # ArgoCD default namespace

  tasks:
    # Kubernetes Namespaces
    - name: Ensure Kubernetes Dev Namespace exists
      kubernetes.core.k8s:
        state: present
        definition:
          apiVersion: v1
          kind: Namespace
          metadata:
            name: "{{ app_namespace_dev }}"

    - name: Ensure Kubernetes Prod Namespace exists
      kubernetes.core.k8s:
        state: present
        definition:
          apiVersion: v1
          kind: Namespace
          metadata:
            name: "{{ app_namespace_prod }}"


    # Harbor Registry Secrets
    - name: Create Image Pull Secret for Harbor Registry
      vars:
        dockerconfig: "{{ {'auths': {harbor_registry: {'username': harbor_user, 'password': harbor_pw, 'auth': (harbor_user + ':' + harbor_pw) | b64encode}}} }}"
      kubernetes.core.k8s:
        state: present
        definition:
          apiVersion: v1
          kind: Secret
          metadata:
            name: harbor-registry-secret
            namespace: "{{ app_namespace_dev }}"
          type: kubernetes.io/dockerconfigjson
          data:
            .dockerconfigjson: "{{ dockerconfig | to_json | b64encode }}"

    - name: Create Image Pull Secret for Harbor Registry
      vars:
        dockerconfig: "{{ {'auths': {harbor_registry: {'username': harbor_user, 'password': harbor_pw, 'auth': (harbor_user + ':' + harbor_pw) | b64encode}}} }}"
      kubernetes.core.k8s:
        state: present
        definition:
          apiVersion: v1
          kind: Secret
          metadata:
            name: harbor-registry-secret
            namespace: "{{ app_namespace_prod }}"
          type: kubernetes.io/dockerconfigjson
          data:
            .dockerconfigjson: "{{ dockerconfig | to_json | b64encode }}"


    # Argo CD GitLab Repository
    - name: Add GitLab Repository via Argo CD Repository Secret
      kubernetes.core.k8s:
        state: present
        definition:
          apiVersion: v1
          kind: Secret
          metadata:
            name: "{{ gitlab_repository_name }}"
            namespace: "{{ argocd_namespace }}"
            labels:
              argocd.argoproj.io/secret-type: repository
          type: Opaque
          stringData:
            url: "{{ gitlab_helm_repository }}"
            sshPrivateKey: "{{ argocd_repository_private_key }}"


    # Argo CD ApplicationSet
    - name: Create Argo CD ApplicationSet
      kubernetes.core.k8s:
        state: present
        apply: true
        wait: true
        definition:
          apiVersion: argoproj.io/v1alpha1
          kind: ApplicationSet
          metadata:
            name: "{{ gitlab_repository_name }}"
            namespace: "{{ argocd_namespace }}"
          spec:
            generators:
              - list:
                  elements:
                    - branch: main
                      namespace: "{{ app_namespace_dev }}"
                    - branch: prod
                      namespace: "{{ app_namespace_prod }}"
            template:
              metadata:
                name: '{{ gitlab_repository_name }}-{% raw %}{{branch}}{% endraw %}'
              spec:
                project: default
                source:
                  repoURL: "{{ gitlab_helm_repository }}"
                  targetRevision: '{% raw %}{{branch}}{% endraw %}'
                  path: helm-chart
                  helm:
                    valueFiles:
                      - values.yaml
                destination:
                  server: https://kubernetes.default.svc
                  namespace: '{% raw %}{{namespace}}{% endraw %}'
                syncPolicy:
                  automated:
                    prune: true
                    selfHeal: true

# Run Ansible playbook: 
ansible-playbook playbooks/argocd_applicationset.yml -i inventory



Argo CD
#

Verify Applications
#

Argo CD Applications:

Main / Dev Application:

Prod Application:



Argo CD CLI: Verify Resources
#

CLI Login
# 

# Login to Argo CD instance
argocd login argocd.jklug.work --grpc-web

# Default user
admin

List GitLab Repositories
#

The GitLab Helm chart repository is connected via the applicationset-1-helm Kubernetes secret, to remove the GitLab repository, delete the Kubernetes secret.

# List GitLab repositories
argocd repo list

# Shell output:
TYPE  NAME  REPO                                                                               INSECURE  OCI    LFS    CREDS  STATUS      MESSAGE  PROJECT
git         git@gitlab.jklug.work:production/argocd-applicationsets/applicationset-1-helm.git  false     false  false  false  Successful

List ApplicationSet
# 

# List Argo CD ApplicationSets
kubectl -n argocd get applicationsets

# Shell output:
NAME                    AGE
applicationset-1-helm   32s

# Delete Argo CD ApplicationSet
kubectl -n argocd delete applicationset applicationset-1-helm

List Application
# 

# Verify Argo CD Applications
argocd app list

# Shell output:
NAME                               CLUSTER                         NAMESPACE         PROJECT  STATUS  HEALTH   SYNCPOLICY  CONDITIONS  REPO                                                                               PATH        TARGET
argocd/applicationset-1-helm-main  https://kubernetes.default.svc  example-app-dev   default  Synced  Healthy  Auto-Prune  <none>      git@gitlab.jklug.work:production/argocd-applicationsets/applicationset-1-helm.git  helm-chart  main
argocd/applicationset-1-helm-prod  https://kubernetes.default.svc  example-app-prod  default  Synced  Healthy  Auto-Prune  <none>      git@gitlab.jklug.work:production/argocd-applicationsets/applicationset-1-helm.git  helm-chart  prod

# List application details: Main / Dev
argocd app get argocd/applicationset-1-helm-main

# Shell output:
Name:               argocd/applicationset-1-helm-main
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          example-app-dev
URL:                https://argocd.jklug.work/applications/applicationset-1-helm-main
Source:
- Repo:             git@gitlab.jklug.work:production/argocd-applicationsets/applicationset-1-helm.git
  Target:           main
  Path:             helm-chart
  Helm Values:      values.yaml
SyncWindow:         Sync Allowed
Sync Policy:        Automated (Prune)
Sync Status:        Synced to main (e523084)
Health Status:      Healthy

GROUP              KIND        NAMESPACE        NAME                 STATUS  HEALTH   HOOK  MESSAGE
                   Service     example-app-dev  example-app-service  Synced  Healthy        service/example-app-service created
apps               Deployment  example-app-dev  example-app          Synced  Healthy        deployment.apps/example-app created
networking.k8s.io  Ingress     example-app-dev  example-app-ingress  Synced  Healthy        ingress.networking.k8s.io/example-app-ingress created

# List application details: Prod
argocd app get argocd/applicationset-1-helm-prod

# Shell output:
Name:               argocd/applicationset-1-helm-prod
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          example-app-prod
URL:                https://argocd.jklug.work/applications/applicationset-1-helm-prod
Source:
- Repo:             git@gitlab.jklug.work:production/argocd-applicationsets/applicationset-1-helm.git
  Target:           prod
  Path:             helm-chart
  Helm Values:      values.yaml
SyncWindow:         Sync Allowed
Sync Policy:        Automated (Prune)
Sync Status:        Synced to prod (bc75e12)
Health Status:      Healthy

GROUP              KIND        NAMESPACE         NAME                 STATUS  HEALTH   HOOK  MESSAGE
                   Service     example-app-prod  example-app-service  Synced  Healthy        service/example-app-service created
apps               Deployment  example-app-prod  example-app          Synced  Healthy        deployment.apps/example-app created
networking.k8s.io  Ingress     example-app-prod  example-app-ingress  Synced  Healthy        ingress.networking.k8s.io/example-app-ingress created



Kubectl
#

Verify Kubernetes Resources
#

Dev
# 

# List default resources
kubectl get all -n example-app-dev

# Shell output:
NAME                               READY   STATUS    RESTARTS   AGE
pod/example-app-65567f4c96-m9zrc   1/1     Running   0          3m38s

NAME                          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/example-app-service   ClusterIP   10.201.53.247   <none>        80/TCP    3m38s

NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/example-app   1/1     1            1           3m38s

NAME                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/example-app-65567f4c96   1         1         1       3m38s

# List Ingress resource
kubectl get ingress -n example-app-dev

# Shell output:
NAME                  CLASS           HOSTS                        ADDRESS          PORTS   AGE
example-app-ingress   nginx-metallb   example-app-dev.jklug.work   192.168.70.110   80      3m48s

Prod
# 

# List default resources
kubectl get all -n example-app-prod

# Shell output:
NAME                               READY   STATUS    RESTARTS   AGE
pod/example-app-6cf8956d48-klwgm   1/1     Running   0          5m20s

NAME                          TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
service/example-app-service   ClusterIP   10.201.91.3   <none>        80/TCP    5m20s

NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/example-app   1/1     1            1           5m20s

NAME                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/example-app-6cf8956d48   1         1         1       5m20s

# List Ingress resource
kubectl get ingress -n example-app-prod

# Shell output:
NAME                  CLASS           HOSTS                         ADDRESS          PORTS   AGE
example-app-ingress   nginx-metallb   example-app-prod.jklug.work   192.168.70.110   80      5m30s