I want to build lxc containers automatically, ideally in a GitLab CI pipeline, but I do not seem to find a way to make it work. I am currently using distrobuilder to build for example a timescale alpine LXC container. That in it self is not really convenient because if I want to upgrade the alpine config there is not streamline way to do it, but for now I am fine with that.
But I can not find a way to run distrobuilder in GitLab CI (so basically in docker) without using privileged runners, and for security reasons this is not an option. Has anybody found a convenient way to automatically build and deploy LXC container in a similar convenient way to docker.
Any help or pointers are greatly appreciated!
1 Answer 1
From https://github.com/oneclickvirt/lxc_amd64_images or https://github.com/oneclickvirt/lxc_arm_images
You can try Github Action, example debian:
name: debian x86_64
on:
schedule:
- cron: '0 12 * * *'
workflow_dispatch:
jobs:
debian_x86_64_images:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: check path
run: |
pwd
- name: Configure Git
run: |
git config --global user.name "daily-update"
git config --global user.email "[email protected]"
- name: Build and Upload Images
run: |
distros=("debian")
for distro in "${distros[@]}"; do
zip_name_list=($(bash build_images.sh $distro false x86_64 | tail -n 1))
release_id=$(curl -s -H "Accept: application/vnd.github.v3+json" "https://api.github.com/repos/oneclickvirt/lxc_amd64_images/releases/tags/$distro" | jq -r '.id')
echo "Building $distro and packge zips"
bash build_images.sh $distro true x86_64
for file in "${zip_name_list[@]}"; do
if [ -f "$file" ] && [ $(stat -c %s "$file") -gt 10485760 ]; then
echo "Checking if $file already exists in release..."
existing_asset_id=$(curl -s -H "Accept: application/vnd.github.v3+json" \
"https://api.github.com/repos/oneclickvirt/lxc_amd64_images/releases/$release_id/assets" \
| jq -r --arg name "$(basename "$file")" '.[] | select(.name == $name) | .id')
if [ -n "$existing_asset_id" ]; then
echo "Asset $file already exists in release, deleting existing asset..."
delete_response=$(curl -s -X DELETE -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" "https://api.github.com/repos/oneclickvirt/lxc_amd64_images/releases/assets/$existing_asset_id")
echo "$delete_response"
if [ $? -eq 0 ] && ! echo "$delete_response" | grep -q "error"; then
echo "Existing asset deleted successfully."
else
echo "Failed to delete existing asset. Skipping file upload..."
rm -rf $file
continue
fi
else
echo "No $file file."
fi
echo "Uploading $file to release..."
curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "Content-Type: application/zip" \
--data-binary @"$file" \
"https://uploads.github.com/repos/oneclickvirt/lxc_amd64_images/releases/$release_id/assets?name=$(basename "$file")"
rm -rf $file
else
echo "No $file or less than 10 MB"
fi
done
done
build_images.sh
#!/bin/bash
run_funct="${1:-debian}"
is_build_image="${2:-false}"
build_arch="${3:-amd64}"
zip_name_list=()
opath=$(pwd)
rm -rf *.tar.xz
ls
if command -v apt-get >/dev/null 2>&1; then
# ubuntu debian kali
if ! command -v sudo >/dev/null 2>&1; then
apt-get install sudo -y
fi
if ! command -v zip >/dev/null 2>&1; then
sudo apt-get install zip -y
fi
if ! command -v jq >/dev/null 2>&1; then
sudo apt-get install jq -y
fi
uname_output=$(uname -a)
if [[ $uname_output != *ARM* && $uname_output != *arm* && $uname_output != *aarch* ]]; then
if ! command -v snap >/dev/null 2>&1; then
sudo apt-get install snapd -y
fi
sudo systemctl start snapd
if ! command -v distrobuilder >/dev/null 2>&1; then
sudo snap install distrobuilder --classic
fi
else
# if ! command -v snap >/dev/null 2>&1; then
# sudo apt-get install snapd -y
# fi
# sudo systemctl start snapd
# if ! command -v distrobuilder >/dev/null 2>&1; then
# sudo snap install distrobuilder --classic
# fi
if ! command -v distrobuilder >/dev/null 2>&1; then
$HOME/goprojects/bin/distrobuilder --version
fi
if [ $? -ne 0 ]; then
sudo apt-get install build-essential -y
export CGO_ENABLED=1
export CC=gcc
wget https://go.dev/dl/go1.21.6.linux-arm64.tar.gz
chmod 777 go1.21.6.linux-arm64.tar.gz
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.21.6.linux-arm64.tar.gz
export GOROOT=/usr/local/go
export PATH=$GOROOT/bin:$PATH
export GOPATH=$HOME/goprojects/
go version
apt-get install -q -y debootstrap rsync gpg squashfs-tools git make
git config --global user.name "daily-update"
git config --global user.email "[email protected]"
mkdir -p $HOME/go/src/github.com/lxc/
cd $HOME/go/src/github.com/lxc/
git clone https://github.com/lxc/distrobuilder
cd ./distrobuilder
make
export PATH=$HOME/goprojects/bin/distrobuilder:$PATH
echo $PATH
find $HOME -name distrobuilder -type f 2>/dev/null
$HOME/goprojects/bin/distrobuilder --version
fi
# wget https://api.ilolicon.com/distrobuilder.deb
# dpkg -i distrobuilder.deb
fi
if ! command -v debootstrap >/dev/null 2>&1; then
sudo apt-get install debootstrap -y
fi
fi
build_or_list_images() {
local versions=()
local ver_nums=()
local variants=()
read -ra versions <<< "1ドル"
read -ra ver_nums <<< "2ドル"
read -ra variants <<< "3ドル"
local architectures=("$build_arch")
local len=${#versions[@]}
for ((i = 0; i < len; i++)); do
version=${versions[i]}
ver_num=${ver_nums[i]}
for arch in "${architectures[@]}"; do
for variant in "${variants[@]}"; do
# apk apt dnf egoportage opkg pacman portage yum equo xbps zypper luet slackpkg
if [[ "$run_funct" == "centos" || "$run_funct" == "fedora" || "$run_funct" == "openeuler" ]]; then
manager="yum"
elif [[ "$run_funct" == "kali" || "$run_funct" == "ubuntu" || "$run_funct" == "debian" ]]; then
manager="apt"
elif [[ "$run_funct" == "almalinux" || "$run_funct" == "rockylinux" || "$run_funct" == "oracle" ]]; then
manager="dnf"
elif [[ "$run_funct" == "archlinux" ]]; then
manager="pacman"
elif [[ "$run_funct" == "alpine" ]]; then
manager="apk"
elif [[ "$run_funct" == "openwrt" ]]; then
manager="opkg"
[ "${version}" = "snapshot" ] && manager="apk"
elif [[ "$run_funct" == "gentoo" ]]; then
manager="portage"
elif [[ "$run_funct" == "opensuse" ]]; then
manager="zypper"
else
echo "Unsupported distribution: $run_funct"
exit 1
fi
EXTRA_ARGS=""
if [[ "$run_funct" == "centos" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
[ "${arch}" = "arm64" ] && arch="aarch64"
if [ "$version" = "7" ] && [ "${arch}" != "amd64" ] && [ "${arch}" != "x86_64" ]; then
EXTRA_ARGS="-o source.url=http://mirror.math.princeton.edu/pub/centos-altarch/ -o source.skip_verification=true"
fi
if [ "$version" = "8-Stream" ] || [ "$version" = "9-Stream" ]; then
EXTRA_ARGS="${EXTRA_ARGS} -o source.variant=boot"
fi
if [ "$version" = "9-Stream" ]; then
EXTRA_ARGS="${EXTRA_ARGS} -o source.url=https://mirror1.hs-esslingen.de/pub/Mirrors/centos-stream"
fi
elif [[ "$run_funct" == "rockylinux" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
[ "${arch}" = "arm64" ] && arch="aarch64"
EXTRA_ARGS="-o source.variant=boot"
elif [[ "$run_funct" == "almalinux" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
[ "${arch}" = "arm64" ] && arch="aarch64"
EXTRA_ARGS="-o source.variant=boot"
elif [[ "$run_funct" == "oracle" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
[ "${arch}" = "arm64" ] && arch="aarch64"
if [[ "$version" == "9" ]]; then
EXTRA_ARGS="-o source.url=https://yum.oracle.com/ISOS/OracleLinux"
fi
elif [[ "$run_funct" == "archlinux" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
[ "${arch}" = "arm64" ] && arch="aarch64"
if [ "${arch}" != "amd64" ] && [ "${arch}" != "i386" ] && [ "${arch}" != "x86_64" ]; then
EXTRA_ARGS="-o source.url=http://os.archlinuxarm.org"
fi
elif [[ "$run_funct" == "alpine" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
[ "${arch}" = "arm64" ] && arch="aarch64"
if [ "${version}" = "edge" ]; then
EXTRA_ARGS="-o source.same_as=3.19"
fi
elif [[ "$run_funct" == "fedora" || "$run_funct" == "openeuler" || "$run_funct" == "opensuse" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
[ "${arch}" = "arm64" ] && arch="aarch64"
elif [[ "$run_funct" == "gentoo" ]]; then
[ "${arch}" = "x86_64" ] && arch="amd64"
[ "${arch}" = "aarch64" ] && arch="arm64"
if [ "${variant}" = "cloud" ]; then
EXTRA_ARGS="-o source.variant=openrc"
else
EXTRA_ARGS="-o source.variant=${variant}"
fi
elif [[ "$run_funct" == "debian" ]]; then
[ "${arch}" = "x86_64" ] && arch="amd64"
[ "${arch}" = "aarch64" ] && arch="arm64"
elif [[ "$run_funct" == "ubuntu" ]]; then
[ "${arch}" = "x86_64" ] && arch="amd64"
[ "${arch}" = "aarch64" ] && arch="arm64"
if [ "${arch}" != "amd64" ] && [ "${arch}" != "i386" ] && [ "${arch}" != "x86_64" ]; then
EXTRA_ARGS="-o source.url=http://ports.ubuntu.com/ubuntu-ports"
fi
fi
if [ "$is_build_image" == true ]; then
if command -v distrobuilder >/dev/null 2>&1; then
if [[ "$run_funct" == "gentoo" ]]; then
echo "sudo distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.architecture=${arch} -o image.variant=${variant} ${EXTRA_ARGS}"
if sudo distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.architecture=${arch} -o image.variant=${variant} ${EXTRA_ARGS}; then
echo "Command succeeded"
fi
elif [[ "$run_funct" != "archlinux" ]]; then
echo "sudo distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.release=${version} -o image.architecture=${arch} -o image.variant=${variant} -o packages.manager=${manager} ${EXTRA_ARGS}"
if sudo distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.release=${version} -o image.architecture=${arch} -o image.variant=${variant} -o packages.manager=${manager} ${EXTRA_ARGS}; then
echo "Command succeeded"
fi
else
echo "sudo distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.architecture=${arch} -o image.variant=${variant} -o packages.manager=${manager} ${EXTRA_ARGS}"
if sudo distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.architecture=${arch} -o image.variant=${variant} -o packages.manager=${manager} ${EXTRA_ARGS}; then
echo "Command succeeded"
fi
fi
else
if [[ "$run_funct" == "gentoo" ]]; then
echo "sudo $HOME/goprojects/bin/distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.architecture=${arch} -o image.variant=${variant} ${EXTRA_ARGS}"
if sudo $HOME/goprojects/bin/distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.architecture=${arch} -o image.variant=${variant} ${EXTRA_ARGS}; then
echo "Command succeeded"
fi
elif [[ "$run_funct" != "archlinux" ]]; then
echo "sudo $HOME/goprojects/bin/distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.release=${version} -o image.architecture=${arch} -o image.variant=${variant} -o packages.manager=${manager} ${EXTRA_ARGS}"
if sudo $HOME/goprojects/bin/distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.release=${version} -o image.architecture=${arch} -o image.variant=${variant} -o packages.manager=${manager} ${EXTRA_ARGS}; then
echo "Command succeeded"
fi
else
echo "sudo $HOME/goprojects/bin/distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.architecture=${arch} -o image.variant=${variant} -o packages.manager=${manager} ${EXTRA_ARGS}"
if sudo $HOME/goprojects/bin/distrobuilder build-lxc "${opath}/images_yaml/${run_funct}.yaml" -o image.architecture=${arch} -o image.variant=${variant} -o packages.manager=${manager} ${EXTRA_ARGS}; then
echo "Command succeeded"
fi
fi
fi
if [[ "$run_funct" == "gentoo" || "$run_funct" == "debian" || "$run_funct" == "ubuntu" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
elif [[ "$run_funct" == "fedora" || "$run_funct" == "openeuler" || "$run_funct" == "opensuse" || "$run_funct" == "alpine" || "$run_funct" == "oracle" || "$run_funct" == "archlinux" ]]; then
[ "${arch}" = "aarch64" ] && arch="arm64"
elif [[ "$run_funct" == "almalinux" || "$run_funct" == "centos" || "$run_funct" == "rockylinux" ]]; then
[ "${arch}" = "aarch64" ] && arch="arm64"
fi
ls
if [ -f rootfs.tar.xz ]; then
mv rootfs.tar.xz "${run_funct}_${ver_num}_${version}_${arch}_${variant}.tar.xz"
rm -rf rootfs.tar.xz
fi
ls
else
if [[ "$run_funct" == "gentoo" || "$run_funct" == "debian" || "$run_funct" == "ubuntu" ]]; then
[ "${arch}" = "amd64" ] && arch="x86_64"
elif [[ "$run_funct" == "fedora" || "$run_funct" == "openeuler" || "$run_funct" == "opensuse" || "$run_funct" == "alpine" || "$run_funct" == "oracle" || "$run_funct" == "archlinux" ]]; then
[ "${arch}" = "aarch64" ] && arch="arm64"
elif [[ "$run_funct" == "almalinux" || "$run_funct" == "centos" || "$run_funct" == "rockylinux" ]]; then
[ "${arch}" = "aarch64" ] && arch="arm64"
fi
zip_name_list+=("${run_funct}_${ver_num}_${version}_${arch}_${variant}.tar.xz")
fi
done
done
done
if [ "$is_build_image" == false ]; then
echo "${zip_name_list[@]}"
fi
}
case "$run_funct" in
debian)
build_or_list_images "buster bullseye bookworm trixie" "10 11 12 13" "default cloud"
;;
ubuntu)
build_or_list_images "bionic focal jammy lunar mantic noble" "18.04 20.04 22.04 23.04 23.10 24.04" "default cloud"
;;
kali)
build_or_list_images "kali-rolling" "latest" "default cloud"
;;
archlinux)
build_or_list_images "current" "current" "default cloud"
;;
gentoo)
build_or_list_images "current" "current" "cloud systemd openrc"
;;
centos)
build_or_list_images "7 8-Stream 9-Stream" "7 8 9" "default cloud"
;;
almalinux)
URL="https://raw.githubusercontent.com/lxc/lxc-ci/main/jenkins/jobs/image-almalinux.yaml"
curl_output=$(curl -s "$URL" | awk '/name: release/{flag=1; next} /^$/{flag=0} flag && /^ *-/{if (!first) {printf "%s", 2ドル; first=1} else {printf " %s", 2ドル}}' | sed 's/"//g')
build_or_list_images "$curl_output" "$curl_output" "default cloud"
;;
rockylinux)
URL="https://raw.githubusercontent.com/lxc/lxc-ci/main/jenkins/jobs/image-rockylinux.yaml"
curl_output=$(curl -s "$URL" | awk '/name: release/{flag=1; next} /^$/{flag=0} flag && /^ *-/{if (!first) {printf "%s", 2ドル; first=1} else {printf " %s", 2ドル}}' | sed 's/"//g')
build_or_list_images "$curl_output" "$curl_output" "default cloud"
;;
alpine)
URL="https://raw.githubusercontent.com/lxc/lxc-ci/main/jenkins/jobs/image-alpine.yaml"
curl_output=$(curl -s "$URL" | awk '/name: release/{flag=1; next} /^$/{flag=0} flag && /^ *-/{if (!first) {printf "%s", 2ドル; first=1} else {printf " %s", 2ドル}}' | sed 's/"//g')
build_or_list_images "$curl_output" "$curl_output" "default cloud"
;;
openwrt)
URL="https://raw.githubusercontent.com/lxc/lxc-ci/main/jenkins/jobs/image-openwrt.yaml"
curl_output=$(curl -s "$URL" | awk '/name: release/{flag=1; next} /^$/{flag=0} flag && /^ *-/{if (!first) {printf "%s", 2ドル; first=1} else {printf " %s", 2ドル}}' | sed 's/"//g')
build_or_list_images "$curl_output" "$curl_output" "default cloud"
;;
oracle)
URL="https://raw.githubusercontent.com/lxc/lxc-ci/main/jenkins/jobs/image-oracle.yaml"
curl_output=$(curl -s "$URL" | awk '/name: release/{flag=1; next} /^$/{flag=0} flag && /^ *-/{if (!first) {printf "%s", 2ドル; first=1} else {printf " %s", 2ドル}}' | sed 's/"//g')
build_or_list_images "$curl_output" "$curl_output" "default cloud"
;;
fedora)
URL="https://raw.githubusercontent.com/lxc/lxc-ci/main/jenkins/jobs/image-fedora.yaml"
curl_output=$(curl -s "$URL" | awk '/name: release/{flag=1; next} /^$/{flag=0} flag && /^ *-/{if (!first) {printf "%s", 2ドル; first=1} else {printf " %s", 2ドル}}' | sed 's/"//g')
build_or_list_images "$curl_output" "$curl_output" "default cloud"
;;
opensuse)
URL="https://raw.githubusercontent.com/lxc/lxc-ci/main/jenkins/jobs/image-opensuse.yaml"
curl_output=$(curl -s "$URL" | awk '/name: release/{flag=1; next} /^$/{flag=0} flag && /^ *-/{if (!first) {printf "%s", 2ドル; first=1} else {printf " %s", 2ドル}}' | sed 's/"//g')
build_or_list_images "$curl_output" "$curl_output" "default cloud"
;;
openeuler)
URL="https://raw.githubusercontent.com/lxc/lxc-ci/main/jenkins/jobs/image-openeuler.yaml"
curl_output=$(curl -s "$URL" | awk '/name: release/{flag=1; next} /^$/{flag=0} flag && /^ *-/{if (!first) {printf "%s", 2ドル; first=1} else {printf " %s", 2ドル}}' | sed 's/"//g')
build_or_list_images "$curl_output" "$curl_output" "default cloud"
;;
*)
echo "Invalid distribution specified."
;;
esac
https://github.com/lxc/lxc-ci/tree/main/images
From here you can find the available yaml files, each corresponding to a class of operating system.
Github Action's official runner supports direct download and compilation of distrobuilder, you can try it yourself.
Sign up to request clarification or add additional context in comments.