↓Skip to main content

Zsync Delta Transfer: Image Distribution with Zsync and Nginx

680 words·
Zsync Nginx
Table of Contents

With Zsync only the difference (delta) between two files is transferred.

Overview & My Setup
#

This tutorial I’m using the following Debian 12 based VMs:

192.168.30.160  # Nginx / Zsync Server
192.168.30.161  # Host used for file download



Prerequisites
#

Install Zsync
#

Install zsync on both servers:

# Install zsync package (debian)
sudo apt install zsync -y



Nginx / Zsync Server
#

Prepare Container Images
#

Pull Docker Images
# 

# Pull some example images
docker pull node:23.2.0
docker pull node:23.6.1
docker pull node:24.0.2

# List Docker images
docker images

# Shell output:
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
node         24.0.2    cc4ee4df2416   2 weeks ago    1.13GB
node         23.6.1    89871f29e084   4 months ago   1.12GB
node         23.2.0    1e7add1c04c7   6 months ago   1.12GB

Save Images as Tarball
# 

# Save the images as tarball
docker save node:23.2.0 > node-23.2.0.tar
docker save node:23.6.1 > node-23.6.1.tar
docker save node:24.0.2 > node-24.0.2.tar

Compress Images with Gzip
# 

# Compress the images with gzip
gzip -k node-23.2.0.tar
gzip -k node-23.6.1.tar
gzip -k node-24.0.2.tar
  • -k Keep the original file
# List files
ls -lh

# Shell output
-rw-r--r-- 1 debian debian 1.1G May 30 19:34 node-23.2.0.tar
-rw-r--r-- 1 debian debian 381M May 30 19:34 node-23.2.0.tar.gz
-rw-r--r-- 1 debian debian 1.1G May 30 19:34 node-23.6.1.tar
-rw-r--r-- 1 debian debian 382M May 30 19:34 node-23.6.1.tar.gz
-rw-r--r-- 1 debian debian 1.1G May 30 19:34 node-24.0.2.tar
-rw-r--r-- 1 debian debian 383M May 30 19:34 node-24.0.2.tar.gz

Generate Zsync Image Metadata
# 

zsyncmake -b 512 node-23.2.0.tar.gz
zsyncmake -b 512 node-23.6.1.tar.gz
zsyncmake -b 512 node-24.0.2.tar.gz

# Verify zsync files
ls -lh

# Shell output:
-rw-r--r-- 1 debian debian 1.1G May 30 19:34 node-23.2.0.tar
-rw-r--r-- 1 debian debian 381M May 30 19:34 node-23.2.0.tar.gz
-rw-r--r-- 1 debian debian  26M May 30 19:42 node-23.2.0.tar.zsync
-rw-r--r-- 1 debian debian 1.1G May 30 19:34 node-23.6.1.tar
-rw-r--r-- 1 debian debian 382M May 30 19:34 node-23.6.1.tar.gz
-rw-r--r-- 1 debian debian  26M May 30 19:42 node-23.6.1.tar.zsync
-rw-r--r-- 1 debian debian 1.1G May 30 19:34 node-24.0.2.tar
-rw-r--r-- 1 debian debian 383M May 30 19:34 node-24.0.2.tar.gz
-rw-r--r-- 1 debian debian  26M May 30 19:42 node-24.0.2.tar.zsync

Host Files With Nginx
#

Nginx Setup
# 

# Create folder structure
sudo mkdir -p /opt/nginx-zsync/zsync-files

# Change owner
sudo chown -R debian:debian /opt/nginx-zsync

# Create Docker Compose manifest
vi /opt/nginx-zsync/docker-compose.yml

# docker-compose.yml
services:
  nginx:
    image: nginx:latest
    container_name: nginx-zsync-server
    ports:
      - "8080:80"
    volumes:
      - ./zsync-files:/usr/share/nginx/html:ro
    restart: unless-stopped

# Start container
cd /opt/nginx-zsync && 
docker compose up -d

Host .tar.gz .zsync Files with Nginx
#

Host the .tar.gz and .zsync files on a web server that supports HTTP range requests for example Nginx:

# Add files to Nginx
cp node-23.2.0.tar.gz /opt/nginx-zsync/zsync-files/
cp node-23.2.0.tar.zsync /opt/nginx-zsync/zsync-files/
cp node-23.6.1.tar.gz /opt/nginx-zsync/zsync-files/
cp node-23.6.1.tar.zsync /opt/nginx-zsync/zsync-files/
cp node-24.0.2.tar.gz /opt/nginx-zsync/zsync-files/
cp node-24.0.2.tar.zsync /opt/nginx-zsync/zsync-files/



Host
#

Download First Image
# 

# Download the first image with zsync (Save to current directory)
zsync http://192.168.30.160:8080/node-23.2.0.tar.zsync

Shell output:

#################### 100.0% 0.0 kBps DONE

No relevent local data found - I will be downloading the whole file. If that's not what you want, CTRL-C out. You should specify the local file is the old version of the file to download with -i (you might have to decompress it with gzip -d first). Or perhaps you just have no data that helps download the file
downloading from http://192.168.30.160:8080/node-23.2.0.tar.gz:
#################### 100.0% 29001.1 kBps DONE

verifying download...checksum matches OK
used 0 local, fetched 399019429

Download Second Image
#

Use previous tarball as seed:

# Download the second image with zsync
zsync -i node-23.2.0.tar.gz http://192.168.30.160:8080/node-23.6.1.tar.zsync

Shell output:

#################### 100.0% 624.0 kBps DONE

reading seed zcat node\-23\.2\.0\.tar\.gz: ****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************Read node-23.2.0.tar.gz. Target 90.4% complete.      ********************************************************************************
downloading from http://192.168.30.160:8080/node-23.6.1.tar.gz:
#################### 100.0% 18720.5 kBps DONE

verifying download...checksum matches OK
used 1037250048 local, fetched 43388669

Only 43388669 Bytes or 41MB were downloaded!


Download Third Image
#

Use previous tarball as seed:

# Download the second image with zsync
zsync -i node-23.6.1.tar.gz http://192.168.30.160:8080/node-24.0.2.tar.zsync

Shell output:

#################### 100.0% 0.0 kBps DONE

reading seed zcat node\-23\.6\.1\.tar\.gz: ****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************Read node-23.6.1.tar.gz. Target 83.3% complete.      **********************************************************************************
reading seed file node-24.0.2.tar.gz.part: ****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************Read node-24.0.2.tar.gz.part. Target 100.0% complete.      *******************************************************************************
downloading from http://192.168.30.160:8080/node-24.0.2.tar.gz:
#################### 100.0% 0.0 kBps DONE

verifying download...checksum matches OK
used 1150860800 local, fetched 17701

Only 0.016 MB were downloaded.