↓Skip to main content

Kubernetes KubeVirt - Run KVM based Virtual Machines in Kubernetes: Deploy CirrOS example VM, Deploy Debian VM with Cloud-init and DataVolume / PersistentVolumeClaim

1488 words·
Kubernetes KubeVirt Kubernetes-Operator KVM CirrOS Debian Cloud-init
Table of Contents

Overview
#

Kubernetes Setup
#

In this tutorial I’m using the following K3s Kubernetes cluster:

NAME      STATUS   ROLES                  AGE   VERSION        INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
ubuntu1   Ready    control-plane,master   20d   v1.30.5+k3s1   192.168.30.10   <none>        Ubuntu 24.04.1 LTS   6.8.0-45-generic   containerd://1.7.21-k3s2
ubuntu2   Ready    worker                 20d   v1.30.5+k3s1   192.168.30.11   <none>        Ubuntu 24.04.1 LTS   6.8.0-45-generic   containerd://1.7.21-k3s2
ubuntu3   Ready    worker                 20d   v1.30.5+k3s1   192.168.30.12   <none>        Ubuntu 24.04.1 LTS   6.8.0-45-generic   containerd://1.7.21-k3s2
ubuntu4   Ready    worker                 20d   v1.30.5+k3s1   192.168.30.13   <none>        Ubuntu 24.04.1 LTS   6.8.0-45-generic   containerd://1.7.21-k3s2

The VMs a running on VMware Workstation Pro, make sure the option “Virtualize Intel VT-x/EPT or AMD-V/RVI” is enabled in the “Processors” section of each VM.


StorageClass
#

I’m using the default K3s StorageClass for the DataVolume / PersistentVolumeClame (PVC):

# List StorageClasses
kubectl get sc

# Shell output:
NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local-path (default)   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  20d

Prerequisites
#

Verify Hardware Virtualization
# 

# Install cpu-checker package
sudo apt install cpu-checker -y

# Test hardware virtualization
sudo kvm-ok

# Shell output:
INFO: /dev/kvm exists
KVM acceleration can be used



Install KubeVirt Operator
#

Install KubeVirt
#

Find the latest version of the GitHub repository:

# Export the latest KubeVirt version
KUBEVIRT_VERSION=$(curl -s https://api.github.com/repos/kubevirt/kubevirt/releases/latest | awk -F '[ \t":]+' '/tag_name/ {print $3}')


# Optional: Print the latest version
echo $KUBEVIRT_VERSION

# Shell output:
v1.3.1

Install the KubeVirt Operator:

# Install KubeVirt Operator & Custom Resource Definitions
kubectl create -f https://github.com/kubevirt/kubevirt/releases/download/${KUBEVIRT_VERSION}/kubevirt-operator.yaml

# Shell output:
namespace/kubevirt created
customresourcedefinition.apiextensions.k8s.io/kubevirts.kubevirt.io created
priorityclass.scheduling.k8s.io/kubevirt-cluster-critical created
clusterrole.rbac.authorization.k8s.io/kubevirt.io:operator created
serviceaccount/kubevirt-operator created
role.rbac.authorization.k8s.io/kubevirt-operator created
rolebinding.rbac.authorization.k8s.io/kubevirt-operator-rolebinding created
clusterrole.rbac.authorization.k8s.io/kubevirt-operator created
clusterrolebinding.rbac.authorization.k8s.io/kubevirt-operator created
deployment.apps/virt-operator created

# Add the KubeVirt Custom Resource (kubevirt-cr)
kubectl create -f https://github.com/kubevirt/kubevirt/releases/download/${KUBEVIRT_VERSION}/kubevirt-cr.yaml

# Shell output:
kubevirt.kubevirt.io/kubevirt created

Verify the Deployment
#

Wait till “kubevirt.kubevirt.io/kubevirt” switches from Deploying to Deployed, this can take a view minutes:

# List all resources in the "kubevirt" namespace
kubectl get all -n kubevirt

# Shell output:
NAME                                   READY   STATUS    RESTARTS   AGE
pod/virt-api-fdbc87c9-9r26x            1/1     Running   0          103s
pod/virt-api-fdbc87c9-zs2th            1/1     Running   0          103s
pod/virt-controller-844699784f-2zrtg   1/1     Running   0          77s
pod/virt-controller-844699784f-rvzfj   1/1     Running   0          77s
pod/virt-handler-gxftv                 1/1     Running   0          77s
pod/virt-handler-lpmrg                 1/1     Running   0          77s
pod/virt-handler-nq2gf                 1/1     Running   0          77s
pod/virt-handler-wzfpb                 1/1     Running   0          77s
pod/virt-operator-74bdf99686-pbfvq     1/1     Running   0          2m25s
pod/virt-operator-74bdf99686-pkn77     1/1     Running   0          2m25s

NAME                                  TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
service/kubevirt-operator-webhook     ClusterIP   10.43.101.76   <none>        443/TCP   106s
service/kubevirt-prometheus-metrics   ClusterIP   None           <none>        443/TCP   106s
service/virt-api                      ClusterIP   10.43.61.20    <none>        443/TCP   106s
service/virt-exportproxy              ClusterIP   10.43.92.50    <none>        443/TCP   106s

NAME                          DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/virt-handler   4         4         4       4            4           kubernetes.io/os=linux   77s

NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/virt-api          2/2     2            2           103s
deployment.apps/virt-controller   2/2     2            2           77s
deployment.apps/virt-operator     2/2     2            2           2m25s

NAME                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/virt-api-fdbc87c9            2         2         2       103s
replicaset.apps/virt-controller-844699784f   2         2         2       77s
replicaset.apps/virt-operator-74bdf99686     2         2         2       2m25s

NAME                            AGE     PHASE
kubevirt.kubevirt.io/kubevirt   2m14s   Deployed

Install Containerized Data Importer (CDI)
# 

# Export the latest CDI version
export VERSION=$(curl -Ls https://github.com/kubevirt/containerized-data-importer/releases/latest | grep -m 1 -o "v[0-9]\.[0-9]*\.[0-9]*")

# Optional: Print the latest version
echo $VERSION

# Shell output:
v1.60.3

# Install CDI
kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-operator.yaml

# Shell output:
namespace/cdi created
customresourcedefinition.apiextensions.k8s.io/cdis.cdi.kubevirt.io created
clusterrole.rbac.authorization.k8s.io/cdi-operator-cluster created
clusterrolebinding.rbac.authorization.k8s.io/cdi-operator created
serviceaccount/cdi-operator created
role.rbac.authorization.k8s.io/cdi-operator created
rolebinding.rbac.authorization.k8s.io/cdi-operator created
deployment.apps/cdi-operator created

# Create a CDI Custom Resource / trigger the operator's deployment of CDI
kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-cr.yaml

# Shell output:
cdi.cdi.kubevirt.io/cdi created

Verify CDI Deployment
# 

# Verify the CDI deployment
kubectl get cdi -n cdi

# Shell output:
NAME   AGE   PHASE
cdi    17s   Deployed

# List CDI pods
kubectl -n cdi get pods

# Shell output:
NAME                               READY   STATUS    RESTARTS   AGE
cdi-apiserver-555ccd5f7b-fqxvc     1/1     Running   0          26s
cdi-deployment-8bf6546cc-9h7fg     1/1     Running   0          26s
cdi-operator-659fd5d79-x68lh       1/1     Running   0          41s
cdi-uploadproxy-6dcd6d454b-smwkq   1/1     Running   0          26s

Install the VirtCTL Binary
#

VirtCTL is used to manage (start, stop,…) the virtual machines:

# Download the binary
curl -Lo virtctl https://github.com/kubevirt/kubevirt/releases/download/${KUBEVIRT_VERSION}/virtctl-${KUBEVIRT_VERSION}-linux-amd64

# Change permissions
chmod +x virtctl

# Move the binary
sudo cp virtctl /usr/local/bin

# Verify the installation / list version
virtctl version

# Shell output:
Client Version: version.Info{GitVersion:"v1.3.1", GitCommit:"ed1e7ae8548d319fa7aacf315ad198f7241287c5", GitTreeState:"clean", BuildDate:"2024-08-22T08:52:25Z", GoVersion:"go1.22.2 X:nocoverageredesign", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{GitVersion:"v1.3.1", GitCommit:"ed1e7ae8548d319fa7aacf315ad198f7241287c5", GitTreeState:"clean", BuildDate:"2024-08-22T10:09:02Z", GoVersion:"go1.22.2 X:nocoverageredesign", Compiler:"gc", Platform:"linux/amd64"}



Deploy CirrOS Example VM
#

Deploy the VM
# 

# Apply a CirrOS based example VM
kubectl apply -f https://kubevirt.io/labs/manifests/vm.yaml

# Shell output:
virtualmachine.kubevirt.io/testvm created

List VMs
# 

# List VMs
kubectl get vm

# Shell output:
NAME     AGE   STATUS    READY
testvm   5s    Stopped   False

Start the VM
# 

# Start the example VM
virtctl start testvm

# Shell output:
VM testvm was scheduled to start

Verify the VM is Running
# 

# Verify the VM has started
kubectl get vm,vmi,pod

# Shell output:
NAME                                AGE   STATUS    READY
virtualmachine.kubevirt.io/testvm   37s   Running   True

NAME                                        AGE   PHASE     IP          NODENAME   READY
virtualmachineinstance.kubevirt.io/testvm   10s   Running   10.42.2.8   ubuntu3    True

NAME                             READY   STATUS    RESTARTS   AGE
pod/virt-launcher-testvm-j64ck   3/3     Running   0          10s

Connect to the VM Console
# 

# Connect to the example VM console
virtctl console testvm

# Shell output:
Successfully connected to testvm console. The escape sequence is ^]

Login with the following credentials:

# Default user:
cirros

# Default password:
gocubsgo

# List the root filesystem
ls /

# Shell output:
bin         home        lib64       mnt         root        tmp
boot        init        linuxrc     old-root    run         usr
dev         initrd.img  lost+found  opt         sbin        var
etc         lib         media       proc        sys         vmlinuz
  • Exit the Console:

Strg + ]


Delete the example VM
# 

# Delete the VM
kubectl delete vm testvm

# Shell output:
virtualmachine.kubevirt.io "testvm" deleted

Verify the VM is deleted:

# List VMs
kubectl get vm

# Shell output:
No resources found in default namespace.



Debian VM with DataVolume
#

Create a DataVolume
# 

# Create a manifest for the DataVolume
vi debian-dv.yaml

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: "debian-dv"
spec:
  source:
    http:
      url: "https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-amd64.qcow2"
  pvc:
    storageClassName: local-path  # Define StorageClass
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: "8Gi"

# Apply the DataVolume
kubectl apply -f debian-dv.yaml

Verify the DataVolume
# 

# List DataVolumes
kubectl get dv

# Shell output:
NAME        PHASE                  PROGRESS   RESTARTS   AGE
debian-dv   WaitForFirstConsumer   N/A                   5s

Create VM
#

Create SSH Key
# 

# Create a SSH key pair
ssh-keygen -t rsa -b 4096

# Copy the public SSH key
cat ~/.ssh/id_rsa.pub

# Shell output:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCzxvJAG3aJVbUUDK2E1NLtT2VfNr6g4CSgcZRMGpHCzfWTyQDg7FPKUu3P8PZxpaDkSABOZ8mflWJdd6uB2xVa01RfRNfTFlkM9/eZUp8HxAw48FjFjrKOhf4eQaEivxpn1HY945ImywoKkp0qWnerN2GGxEL1zuhlPB8g8fTfl30WUkkat6XR6V7CLEQo0xfBQwpsbohHZw2OahaBPcPyPOtNDibVtexckgAzNrdxBN3vhwhSjtAbr9umcuI7xrZDaQ0IOzxaCwhQLmJnvMn94XqnFQXAMme9drVc7T+k7F21fy3aFs5ZoSu7J+Qk1GPiI8H4xHmG3fyOIozMARl2gqKo0WUjUn0GztYmFFBEnbUe8NW2xSrFoyEEx9pqHUixEZgV9vcNAyX7B97O12yVEB1/HMqDgWQ8I3vLi9kuFdIAtb6SILVaYOinXUi3tkATlebhay9aDeJmYlVthWpsbWH0/kS/DZf922lGXTyyGusyRgzYPY6lurR7E9f/4rcl0o9tXoUjCpnN1CfqhDMuuEK7kCW7ctWkq3wIXhToCCPoNsk7Q6GrkLFuMzDl4mA/K7rf1FLJz2DDMTU8ALA5UMSgyiLGA9W2CJMArzyiVpLM8n8c/HGthuGE5A4Vw5BPjeLkmSG4+neKaMU0Nw2DV7ZWidbWuxxuCuSE5dF4Jw== ubuntu@ubuntu1

Deploy VM
# 

# Create a manifest for the Debian VM
vi debian-vm.yaml

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  labels:
    kubevirt.io/os: linux
  name: debian
spec:
  running: true
  template:
    metadata:
      creationTimestamp: null
      labels:
        kubevirt.io/domain: debian
    spec:
      domain:
        cpu:
          cores: 1
        devices:
          disks:
          - disk:
              bus: virtio
            name: disk0
          - cdrom:
              bus: sata
              readonly: true
            name: cloudinitdisk
        resources:
          requests:
            memory: 512M
      volumes:
      - name: disk0
        persistentVolumeClaim:
          claimName: debian-dv
      - cloudInitNoCloud:
          userData: |
            #cloud-config
            system_info:
              default_user:
                name: debian
                home: /home/debian
            password: my-secure-pw
            chpasswd: { expire: False }
            hostname: debian01
            ssh_pwauth: True
            disable_root: false
            ssh_authorized_keys:
            - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCzxvJAG3aJVbUUDK2E1NLtT2VfNr6g4CSgcZRMGpHCzfWTyQDg7FPKUu3P8PZxpaDkSABOZ8mflWJdd6uB2xVa01RfRNfTFlkM9/eZUp8HxAw48FjFjrKOhf4eQaEivxpn1HY945ImywoKkp0qWnerN2GGxEL1zuhlPB8g8fTfl30WUkkat6XR6V7CLEQo0xfBQwpsbohHZw2OahaBPcPyPOtNDibVtexckgAzNrdxBN3vhwhSjtAbr9umcuI7xrZDaQ0IOzxaCwhQLmJnvMn94XqnFQXAMme9drVc7T+k7F21fy3aFs5ZoSu7J+Qk1GPiI8H4xHmG3fyOIozMARl2gqKo0WUjUn0GztYmFFBEnbUe8NW2xSrFoyEEx9pqHUixEZgV9vcNAyX7B97O12yVEB1/HMqDgWQ8I3vLi9kuFdIAtb6SILVaYOinXUi3tkATlebhay9aDeJmYlVthWpsbWH0/kS/DZf922lGXTyyGusyRgzYPY6lurR7E9f/4rcl0o9tXoUjCpnN1CfqhDMuuEK7kCW7ctWkq3wIXhToCCPoNsk7Q6GrkLFuMzDl4mA/K7rf1FLJz2DDMTU8ALA5UMSgyiLGA9W2CJMArzyiVpLM8n8c/HGthuGE5A4Vw5BPjeLkmSG4+neKaMU0Nw2DV7ZWidbWuxxuCuSE5dF4Jw==            
        name: cloudinitdisk

# Deploy the VM
kubectl apply -f debian-vm.yaml

Verify the DataVolume & PVC
# 

# List DataVolume & PVC
kubectl get dv,pvc


# Shell output:
NAME                                   PHASE      PROGRESS   RESTARTS   AGE
datavolume.cdi.kubevirt.io/debian-dv   PVCBound   N/A                   6m39s

NAME                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/debian-dv   Bound    pvc-40b4d269-e259-4ff5-8d65-20bc13955ec0   8Gi        RWO            local-path     <unset>                 6m39s


# Shell output:
NAME                                   PHASE              PROGRESS   RESTARTS   AGE
datavolume.cdi.kubevirt.io/debian-dv   ImportInProgress   18.76%                6m55s

NAME                                      STATUS   VOLUME                                     CAPACITY     ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/debian-dv           Bound    pvc-40b4d269-e259-4ff5-8d65-20bc13955ec0   8Gi          RWO            local-path     <unset>                 6m55s
persistentvolumeclaim/debian-dv-scratch   Bound    pvc-627331ec-7b11-4905-be53-43fbcd57ce73   8589934592   RWO            local-path     <unset>                 10s


# Shell output: (Wait till PHASE succeeds)
NAME                                   PHASE       PROGRESS   RESTARTS   AGE
datavolume.cdi.kubevirt.io/debian-dv   Succeeded   100.0%                7m16s

NAME                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/debian-dv   Bound    pvc-40b4d269-e259-4ff5-8d65-20bc13955ec0   8Gi        RWO            local-path     <unset>                 7m16s

Verify the VM
# 

# Verify the VM has started
kubectl get vm,vmi

# Shell output:
NAME                                AGE   STATUS    READY
virtualmachine.kubevirt.io/debian   73s   Running   True

NAME                                        AGE   PHASE     IP           NODENAME   READY
virtualmachineinstance.kubevirt.io/debian   73s   Running   10.42.2.65   ubuntu3    True

# List pods
kubectl get pods -o wide

# Shell output:
NAME                         READY   STATUS    RESTARTS   AGE   IP           NODE      NOMINATED NODE   READINESS GATES
virt-launcher-debian-htk57   2/2     Running   0          46s   10.42.2.65   ubuntu3   <none>           1/1

Access the VM
#

Connect to the VM Console
# 

# Connect to the example VM console
virtctl console debian

# Shell output:
Successfully connected to debian console. The escape sequence is ^]

Login with the following credentials:

# Cloud-init defined user:
debian

# Cloud-init defined password:
my-secure-pw
  • Exit the Console:

Strg + ]


SSH Into VM
# 

# Export VM IP
IP=$(kubectl get vmi debian -o jsonpath='{.status.interfaces[0].ipAddress}')

# Optional: Print the VM IP
echo $IP

# SSH into the VM
ssh debian@${IP}

# List the current directory
pwd

# Shell output:
/home/debian

# Exit the SSH connection
exit

Stop the VM
# 

# Stop VM
kubectl patch virtualmachine debian --type='json' -p='[{"op": "replace", "path": "/spec/running", "value": false}]'

# Verify the VM status
kubectl get vm

# Shell output:
NAME     AGE     STATUS    READY
debian   6m20s   Stopped   False

Delete the VM
# 

# Delete the VM
kubectl delete vm debian

Delete the DataVolume
# 

# Delete the DataVolume
kubectl delete dv debian-dv

Verify the DV was deleted:

# List DataVolumes
kubectl get dv

# Shell output:
No resources found in default namespace.

Verify the PVC was deleted:

# List PVCs
kubectl get pvc

# Shell output:
No resources found in default namespace



Links # 

# GitHub Repository
https://github.com/kubevirt/kubevirt