Kubernetes Jobs: Jobs Overview, Basic Non-parallel & Parallel Job Examples; CronJob & RBAC Example that Restarts a Deployment

NAME      STATUS   ROLES           AGE    VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE           KERNEL-VERSION     CONTAINER-RUNTIME
ubuntu1   Ready    control-plane   104d   v1.28.11   192.168.30.10   <none>        Ubuntu 24.04 LTS   6.8.0-36-generic   containerd://1.7.18
ubuntu2   Ready    worker          104d   v1.28.11   192.168.30.11   <none>        Ubuntu 24.04 LTS   6.8.0-36-generic   containerd://1.7.18
ubuntu3   Ready    worker          104d   v1.28.11   192.168.30.12   <none>        Ubuntu 24.04 LTS   6.8.0-36-generic   containerd://1.7.1

Kubernetes Jobs Explanation
#

Overview
#

A Job creates pods that run short-term, one-off tasks that should run once, until completion, or a limited number of times.
When the task is done, the Job finishes.
Jobs are used to process data, run a script, or clean up temporary files.

Job Types
#

Non-Parallel Jobs:

Are designed to run a single task to completion.
Are used for tasks that can’t be divided among multiple pods.
For example a script that needs to be run once a day.

Parallel Jobs with a Fixed Completion Count:

Used to divide the a among several pods, each processing a portion of the data.
The workload can be spread across multiple nodes.
The job is considered complete when a specified number of Pods successfully complete their tasks.

CronJob
#

A CronJob allows to define a job that will run automatically at specified times or intervals.

Basic Example Jobs
#

Create Namespace
#

# Create a new namespace
kubectl create ns example-jobs

Non-parallel Job
#

Create Job
#

# Create a manifest for the job
vi exjob-non-parallel.yaml

Without automatic cleanup:

apiVersion: batch/v1
kind: Job
metadata:
  name: exjob-non-parallel
  namespace: example-jobs
spec:
  template:
    metadata:
      name: job-pod
    spec:
      containers:
      - name: busybox
        image: busybox
        command: ["echo", "print some text"]
      restartPolicy: OnFailure
  completions: 3

Add automatic cleanup:

apiVersion: batch/v1
kind: Job
metadata:
  name: exjob-non-parallel
  namespace: example-jobs
spec:
  template:
    metadata:
      name: job-pod
    spec:
      containers:
      - name: busybox
        image: busybox
        command: ["echo", "print some text"]
      restartPolicy: OnFailure
  ttlSecondsAfterFinished: 10 # Job and pods will be deleted
  completions: 3

# Deploy the manifest
kubectl apply -f exjob-non-parallel.yaml

The automatic cleanup deletes the job and its pods a certain amount of time after the job completes.

Verify Job
#

Verify job without automatic cleanup:

# List pods and jobs in "example-jobs" namespace
kubectl get pod,job -n example-jobs

# Shell output:
NAME                           READY   STATUS      RESTARTS   AGE
pod/exjob-non-parallel-68ds7   0/1     Completed   0          6s
pod/exjob-non-parallel-6n8jn   0/1     Completed   0          14s
pod/exjob-non-parallel-kcvwv   0/1     Completed   0          10s

NAME                           STATUS     COMPLETIONS   DURATION   AGE
job.batch/exjob-non-parallel   Complete   3/3           12s        14s

Verify job with automatic cleanup:

# List pods and jobs in "example-jobs" namespace
kubectl get pod,job -n example-jobs

# Shell output:
NAME                           READY   STATUS      RESTARTS   AGE
pod/exjob-non-parallel-2hffr   0/1     Completed   0          8s
pod/exjob-non-parallel-tgtrj   0/1     Completed   0          12s
pod/exjob-non-parallel-v5kt6   0/1     Completed   0          16s

NAME                           STATUS     COMPLETIONS   DURATION   AGE
job.batch/exjob-non-parallel   Complete   3/3           12s        16s

# Shell output: (Wait 10 seconds)
No resources found in example-jobs namespace.

Delete Job
#

If the job was deployed without automatic cleanup, manually delete the job:

# Delete Job after it's finished
kubectl delete job exjob-non-parallel -n example-jobs

Parallel Job
#

Create Job
#

# Create a manifest for the job
vi exjob-parallel.yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: exjob-parallel
  namespace: example-jobs
spec:
  template:
    metadata:
      name: job-pod
    spec:
      containers:
      - name: busybox
        image: busybox
        command: ["echo", "print some text"]
      restartPolicy: OnFailure
  ttlSecondsAfterFinished: 15 # Job and pods will be deleted
  completions: 12
  parallelism: 3

# Deploy the manifest
kubectl apply -f exjob-parallel.yaml

completions: 12 Specifies the total number of successful completions required for the Job to be considered finished.
parallelism: 3 Defines how many pods can run in parallel at the same time. Kubernetes will start 3 pods in parallel to work on the Job.

Verify Job
#

Verify job without automatic cleanup:

# List pods and jobs in "example-jobs" namespace
kubectl get pod,job -n example-jobs

# Shell output:
NAME                       READY   STATUS      RESTARTS   AGE
pod/exjob-parallel-2v7bd   0/1     Completed   0          4s
pod/exjob-parallel-67hwz   0/1     Completed   0          8s
pod/exjob-parallel-jlw4p   0/1     Completed   0          8s
pod/exjob-parallel-mfqnk   0/1     Completed   0          3s
pod/exjob-parallel-q4q64   0/1     Completed   0          3s
pod/exjob-parallel-xd9w2   0/1     Completed   0          8s

NAME                       STATUS    COMPLETIONS   DURATION   AGE
job.batch/exjob-parallel   Running   3/12          8s         8s


# Shell output:
NAME                       READY   STATUS      RESTARTS   AGE
pod/exjob-parallel-2v7bd   0/1     Completed   0          13s
pod/exjob-parallel-5xgj9   0/1     Completed   0          5s
pod/exjob-parallel-67hwz   0/1     Completed   0          17s
pod/exjob-parallel-ffdcc   0/1     Completed   0          9s
pod/exjob-parallel-jlw4p   0/1     Completed   0          17s
pod/exjob-parallel-ks7vv   0/1     Completed   0          9s
pod/exjob-parallel-mfqnk   0/1     Completed   0          12s
pod/exjob-parallel-n8hvd   0/1     Completed   0          5s
pod/exjob-parallel-q4q64   0/1     Completed   0          12s
pod/exjob-parallel-qjphj   0/1     Completed   0          5s
pod/exjob-parallel-xd9w2   0/1     Completed   0          17s
pod/exjob-parallel-zqn5b   0/1     Completed   0          9s

NAME                       STATUS     COMPLETIONS   DURATION   AGE
job.batch/exjob-parallel   Complete   12/12         16s        17s


# Shell output:
No resources found in example-jobs namespace

CronJob: Restart a Deployment
#

Create Namespace
#

# Create a new namespace
kubectl create ns example-namespace

Example Deployment
#

# Create deployment manifest
vi nginx-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment  # Should match CronJob
  namespace: example-namespace
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

# Apply the manifest
kubectl apply -f nginx-deployment.yaml

ServiceAccount, Role & RoleBinding
#

# Create a manifest for the job permissions
vi permissions.yaml

# ServiceAccount
kind: ServiceAccount
apiVersion: v1
metadata:
  name: restart-deployment
  namespace: example-namespace # Define namespace

---
# Role
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: restart-deployment
  namespace: example-namespace # Define namespace
rules:
  - apiGroups: ["apps", "extensions"]
    resources: ["deployments"]
    resourceNames: ["nginx-deployment"] # Deployment name
    verbs: ["get", "patch", "list", "watch"] # Permissions required for restarting the deployment

---
# RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: restart-deployment
  namespace: example-namespace # Define namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: restart-deployment
subjects:
  - kind: ServiceAccount
    name: restart-deployment
    namespace: example-namespace # Define namespace

# Apply the manifest
kubectl apply -f permissions.yaml

CronJob
#

The following CronJob runs every two minutes:

# Create a manifest for the CronJob
vi restart-cronjob.yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: restart-deployment
  namespace: example-namespace # Define namespace
spec:
  successfulJobsHistoryLimit: 1
  failedJobsHistoryLimit: 2
  concurrencyPolicy: Forbid
  schedule: '*/2 * * * *'
  jobTemplate:
    spec:
      backoffLimit: 2
      activeDeadlineSeconds: 600
      template:
        spec:
          serviceAccountName: restart-deployment
          restartPolicy: Never
          containers:
            - name: kubectl
              image: bitnami/kubectl
              command:
                - 'kubectl'
                - 'rollout'
                - 'restart'
                - 'deployment/nginx-deployment' # Deployment name

# Apply the manifest
kubectl apply -f restart-cronjob.yaml

successfulJobsHistoryLimit: 1 Specifies the number of successful jobs that Kubernetes will keep in the history after their completion. Older successful job records will be deleted.
failedJobsHistoryLimit: 2 Specifies the number of failed jobs that Kubernetes will retain in the history.
concurrencyPolicy: Forbid Defines how Kubernetes handles the execution of jobs if a new job is scheduled to run before the previous one finishes. Forbid Ensures that only one job runs at a time. If a job is still running when a new one is scheduled, the new job will be skipped. Other options are Allow and Replace.
backoffLimit: 2 Specifies how many times Kubernetes will retry a failed job before it is marked as failed and not retried anymore.
activeDeadlineSeconds: 600 mMximum duration (in seconds) that a job is allowed to run before Kubernetes terminates it.

Verify Cronjob & Job
#

# List cronjobs in "example-namespace" namespace
kubectl get cronjob -n example-namespace

# Shell output:
NAME                 SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
restart-deployment   */2 * * * *   False     0        <none>          7s

# List cronjob details
kubectl describe cronjob restart-deployment -n example-namespace

# List jobs created by your CronJob
kubectl get jobs -n example-namespace

# Shell output:
NAME                          COMPLETIONS   DURATION   AGE
restart-deployment-28821114   1/1           3s         4s

Verify the Deployment Restart
#

# List pods in "example-namespace" namespace
kubectl get pods -n example-namespace --watch

# Shell output:
NAME                               READY   STATUS    RESTARTS   AGE
nginx-deployment-dd4dc744c-9fp7p   1/1     Running   0          27s
restart-deployment-28821114-bnft5   0/1     Pending   0          0s
restart-deployment-28821114-bnft5   0/1     Pending   0          0s
restart-deployment-28821114-bnft5   0/1     ContainerCreating   0          0s
nginx-deployment-8fd995c7b-tw7d6    0/1     Pending             0          0s
nginx-deployment-8fd995c7b-tw7d6    0/1     Pending             0          0s
nginx-deployment-8fd995c7b-tw7d6    0/1     ContainerCreating   0          0s
restart-deployment-28821114-bnft5   0/1     Completed           0          1s
restart-deployment-28821114-bnft5   0/1     Completed           0          2s
restart-deployment-28821114-bnft5   0/1     Completed           0          3s
nginx-deployment-8fd995c7b-tw7d6    1/1     Running             0          2s
nginx-deployment-dd4dc744c-9fp7p    1/1     Terminating         0          62s
restart-deployment-28821114-bnft5   0/1     Completed           0          3s
nginx-deployment-dd4dc744c-9fp7p    0/1     Terminating         0          63s
nginx-deployment-dd4dc744c-9fp7p    0/1     Terminating         0          63s
nginx-deployment-dd4dc744c-9fp7p    0/1     Terminating         0          63s
nginx-deployment-dd4dc744c-9fp7p    0/1     Terminating         0          63s
restart-deployment-28821116-vt9r4   0/1     Pending             0          0s
restart-deployment-28821116-vt9r4   0/1     Pending             0          0s
restart-deployment-28821116-vt9r4   0/1     ContainerCreating   0          0s
nginx-deployment-95949fd86-mpf6j    0/1     Pending             0          0s
nginx-deployment-95949fd86-mpf6j    0/1     Pending             0          0s
nginx-deployment-95949fd86-mpf6j    0/1     ContainerCreating   0          0s
restart-deployment-28821116-vt9r4   0/1     Completed           0          2s
restart-deployment-28821116-vt9r4   0/1     Completed           0          3s
restart-deployment-28821116-vt9r4   0/1     Completed           0          4s
nginx-deployment-95949fd86-mpf6j    1/1     Running             0          3s
nginx-deployment-8fd995c7b-tw7d6    1/1     Terminating         0          2m3s
restart-deployment-28821116-vt9r4   0/1     Completed           0          4s
restart-deployment-28821114-bnft5   0/1     Terminating         0          2m4s
restart-deployment-28821114-bnft5   0/1     Terminating         0          2m4s
nginx-deployment-8fd995c7b-tw7d6    0/1     Terminating         0          2m3s
nginx-deployment-8fd995c7b-tw7d6    0/1     Terminating         0          2m4s
nginx-deployment-8fd995c7b-tw7d6    0/1     Terminating         0          2m4s
nginx-deployment-8fd995c7b-tw7d6    0/1     Terminating         0          2m4s

# List deployment details
kubectl describe deployment nginx-deployment -n example-namespa

# Shell output:
Events:
  Type    Reason             Age    From                   Message
  ----    ------             ----   ----                   -------
  Normal  ScalingReplicaSet  6m31s  deployment-controller  Scaled up replica set nginx-deployment-7c79c4bf97 to 1
  Normal  ScalingReplicaSet  5m51s  deployment-controller  Scaled up replica set nginx-deployment-86d6c9f578 to 1
  Normal  ScalingReplicaSet  5m49s  deployment-controller  Scaled down replica set nginx-deployment-7c79c4bf97 to 0 from 1
  Normal  ScalingReplicaSet  4m51s  deployment-controller  Scaled up replica set nginx-deployment-dd4dc744c to 1
  Normal  ScalingReplicaSet  4m49s  deployment-controller  Scaled down replica set nginx-deployment-86d6c9f578 to 0 from 1
  Normal  ScalingReplicaSet  3m51s  deployment-controller  Scaled up replica set nginx-deployment-8fd995c7b to 1
  Normal  ScalingReplicaSet  3m49s  deployment-controller  Scaled down replica set nginx-deployment-dd4dc744c to 0 from 1
  Normal  ScalingReplicaSet  111s   deployment-controller  Scaled up replica set nginx-deployment-95949fd86 to 1
  Normal  ScalingReplicaSet  108s   deployment-controller  Scaled down replica set nginx-deployment-8fd995c7b to 0 from

Delete Resources
#

# Delete the Cronjob
kubectl delete cronjob restart-deployment -n example-namespace

# Delete the example deployment
kubectl delete deployment nginx-deployment -n example-namespace

# Delete the RoleBinding
kubectl delete rolebinding restart-deployment -n example-namespace

# Delete the Role
kubectl delete role restart-deployment -n example-namespace

# Delete the ServiceAccount
kubectl delete serviceaccount restart-deployment -n example-namespace