GitLab CI Pipeline: Cross-Compilation with multiarch/crossbuild Container Image of a Go Application for both x86 and ARM based CPU Architecture, Deployment via SCP

Table of Contents

GitLab-CI-Go-Application - This article is part of a series.

Part 1: GitLab CI Pipeline: Compile and Deploy a Go Application via SCP (Embedded Device Deployment)

Part 2: This Article

Overview
#

In this tutorial I’m using the following setup based on Ubuntu 22.04 servers, GitLab is dockerized:

192.168.70.4 # GitLab
192.168.70.5 # GitLab Runner
192.168.70.6 # Deployment Server
3.70.184.108 # Temporary ARM based Server for execution of the ARM binary

I’m using the same setup for the deployment via SSH as described in my blog post: GitLab CI Pipeline - Deploy website with Nginx Docker Container

The main requirements for the deployment via SSH key are the following:

Create a SSH key pair on the server where GitLab is deployed
Copy the public SSH key to the authorized keys file of the Deployment server
SSH into the deployment servers to add the fingerprint
Add the private SSH key as variable to the GitLab Repository CI/CD settings
On the deployment server, a user named “gitlab-deployment” is used for the authentication of for the GitLab CI pipeline

Deployment Server Prerequisites
#

Deployment Folder
#

Create a folder for the Go binary deployment:

# Create folder
sudo mkdir /go-binary

# Change the permissions
sudo chown -R gitlab-deployment:root /go-binary

GitLab Repository
#

The following GitLab CI pipeline compiles a Go application for both x86 and ARM based CPU architectures.

To keep it simple, I deploy both the x86 and ARM binary on the same deployment server.

After the deployment I manually copy the ARM binary on a temporary AWS based ARM server to verify the ARM binary is executable.

This GitLab repository is available on GitHub:
https://github.com/jueklu/cross-compilation-go-binary

File & Folder Structure
#

The file and folder structure of the GitLab repository looks like this:

GitLab-Repository
├── .gitlab-ci.yml
├── .gitlab-compile.yml
├── go.mod
├── main.go
└── README.md

Pipeline Manifests
#

CI Main Pipeline Manifest
#

.gitlab-ci.yml

# Variables that will be reused in the pipeline
variables:
  DEPLOY_IP: "192.168.70.6"
  DEPLOY_USER: "gitlab-deployment"
  OUTPUT_DIR: go-bin  # Directory to store the compiled binary
  OUTPUT_NAME_AMD64: $OUTPUT_DIR/example-binary_amd64_${CI_COMMIT_REF_SLUG}
  OUTPUT_NAME_ARM64: $OUTPUT_DIR/example-binary_arm64_${CI_COMMIT_REF_SLUG}


# Pipeline stages
stages:
  - compile
  - deploy


### CI Pipeline Parts
include:
  - local: .gitlab-compile.yml


### Deploy Job
deploy:
  stage: deploy
  image: alpine:latest
  before_script:
    # Update the package index, install the OpenSSH client for SSH connections
    - apk update && apk add openssh-client
    # If the private SSH key file ($ID_RSA) exists, set secure permissions (read/write for the owner only)
    - if [ -f "$ID_RSA" ]; then chmod og= $ID_RSA; fi
  script:
    # Copy the binary to the target Linux server
    - scp -i $ID_RSA -o StrictHostKeyChecking=no $OUTPUT_NAME_ARM64 $DEPLOY_USER@$DEPLOY_IP:/go-binary
    - scp -i $ID_RSA -o StrictHostKeyChecking=no $OUTPUT_NAME_AMD64 $DEPLOY_USER@$DEPLOY_IP:/go-binary
  only:
    - main

CI Compilation Manifest
#

.gitlab-compile.yml

### Compile for arm64
compile-arm64:
  image: multiarch/crossbuild
  stage: compile
  variables:
    GOARCH: arm64 # Exported environment variables within the job's execution environment
    GOOS: linux # Exported environment variables within the job's execution environment
  before_script:
    # Install Go
    - apt-get update && apt-get install -y wget
    - wget https://go.dev/dl/go1.23.3.linux-amd64.tar.gz
    - tar -C /usr/local -xzf go1.23.3.linux-amd64.tar.gz
    - export PATH=$PATH:/usr/local/go/bin

    # Create directory for the compiled binary
    - mkdir -p $OUTPUT_DIR

    # Initialize a Go module if it doesn't already exist
    - go mod init example-project || true
    # Download dependencies
    - go mod tidy
  script:
    # Compile the Go binary for arm64
    - go build -o $OUTPUT_NAME_ARM64 ./main.go
    # List the binary
    - ls -la $OUTPUT_DIR
  artifacts:
    paths:
      # Save the binary as an artifact
      - $OUTPUT_NAME_ARM64
  only:
    - main


### Compile for amd64
compile-amd64:
  image: golang:latest
  stage: compile
  before_script:  # Initialize the Go module environment
    # Create directory for the compiled binary
    - mkdir -p $OUTPUT_DIR
    # Initialize a Go module if it doesn't already exist
    - go mod init example-project || true
    # Download dependencies (even if there are none, it's safe to run)
    - go mod tidy
  script:  # Compile the Go binary and store it in the specified output directory
    # Compile the Go binary for amd64 (default architecture)
    - go build -o $OUTPUT_NAME_AMD64 ./main.go
    # List the binary
    - ls -la $OUTPUT_DIR
  artifacts:
    paths:
      # Save the binary as an artifact
      - $OUTPUT_NAME_AMD64
  only:
    - main

Notes
#

Architecture variables: The go build command picks them up because the Go toolchain natively checks for these environment variables to determine the target architecture and operating system.

compile-arm64:
  variables:
    GOARCH: arm64 # Exported environment variables within the job's execution environment
    GOOS: linux # Exported environment variables within the job's execution environment

Go Source Code
#

This Go code will just output “Hi there!” when executed.

main.go
#

main.go

// Declare the main package / required for any standalone executable Go program
package main

// Import the fmt package for formatted input/output functions
import "fmt"

// Define the main function, the entry point of the program
func main() {
  fmt.Println("Hi there!") // Print the message to the console
}

go.mod
#

go.mod

module jueklu/some-go-project

go 1.21.8

Verify the Deployment
#

Verify Deployed Binaries
#

# List the binaries on the deployment server
sudo ls -la /go-binary

# Shell output:
-rwxr-xr-x  1 gitlab-deployment gitlab-deployment 2130778 Nov 18 12:19 example-binary_amd64_main
-rwxr-xr-x  1 gitlab-deployment gitlab-deployment 2154559 Nov 18 12:19 example-binary_arm64_main

Verify AMD64 Binary
#

# Run the Go binary that was compiles for AMD64 architecture:
sudo /go-binary/example-binary_amd64_main

# Shell output:
Hi there!

Test the ARM binary on the AMD64 based server:

# Run the Go binary that was compiles for ARM64 architecture:
sudo /go-binary/example-binary_arm64_main

# Shell output: (Supposed to fail)
-bash: ./example-binary_arm64_main: cannot execute binary file: Exec format error

Verify ARM Binary
#

Note: To keep it simple, I deploy both binaries to the same deployment server. To verify that the ARM64-based binary runs correctly, I copy it to a temporary AWS-based ARM64 Ubuntu server.

Copy Binary
#

Copy the ARM binary to ARM based Server:

# Copy the binary to an ARM based VM
scp -i ~/.ssh/aws-ec2-pc-wien.pem example-binary_arm64_main  ubuntu@3.70.184.108:/home/ubuntu/

# SSH into the ARM based VM
ssh -i ~/.ssh/aws-ec2-pc-wien.pem ubuntu@3.70.184.108

Verify the ARM architecture of the server:

# List hardware architecture
uname -m

# Shell output:
aarch64

aarch64 ARM64 based CPU

Run Binary
#

# Run the ARM binary
./example-binary_arm64_main

# Shell output:
Hi there!

Links
#

# Find latest Go release
https://go.dev/dl/

GitLab-CI-Go-Application - This article is part of a series.

Part 1: GitLab CI Pipeline: Compile and Deploy a Go Application via SCP (Embedded Device Deployment)

Part 2: This Article

Overview #

Deployment Server Prerequisites #

Deployment Folder #

GitLab Repository #

File & Folder Structure #

Pipeline Manifests #

CI Main Pipeline Manifest #

CI Compilation Manifest #

Notes #

Go Source Code #

main.go #

go.mod #

Verify the Deployment #

Verify Deployed Binaries #

Verify AMD64 Binary #

Verify ARM Binary #

Copy Binary #

Run Binary #

Links #