[unix_distributions][fedora] Use chunkah to optimize bootc container updates

[LukeShortCloud]

Side note: bootc notes should be refactored into the [virtualization][containers] notes now that it is distro-agnostic.

chunkah helps to speed up updates by reorganizing layers to contain related files. No infrastructure changes are needed.

Steps to use chunkah:

1. As the last RUN command in a container, add file attribute tags for related files (it is recommended to tag all files from each individual package as being related).
2. Run chunkah on the built container to reorganize the layers based on those file attributes.

https://github.com/coreos/chunkah

Example of step 1 for different Lnux distributions:

Arch Linux:

pacman -Qq | while read -r pkgname; do
  echo "Setting file attributes for package: $pkgname (chunkah)"
  pacman -Qlq "$pkgname" | while read -r filepath; do
    if [[ -f "$filepath" ]]; then
       setfattr -n user.component -v "$pkgname" "$filepath" || true
    fi
  done
done

Debian:

# Set file attributes for packages for chunkah to handle.
dpkg-query -W -f='${Package}\n' | while read -r pkgname; do
  echo "Setting file attributes for package: $pkgname (chunkah)"
  dpkg -L "$pkgname" | while read -r filepath; do
    if [[ -f "$filepath" ]]; then
      setfattr -n user.component -v "$pkgname" "$filepath" || true
    fi
  done
done

https://github.com/frostyard/snosi/blob/09d22407f138307f94b6440fac1d3484a53a5bd3/shared/outformat/image/finalize/mkosi.finalize.chroot#L41

Fedora:

rpm -qa | while read -r pkgname; do
  echo "Setting file attributes for package: $pkgname (chunkah)"
  rpm -ql "$pkgname" | while read -r filepath; do
    if [[ -f "$filepath" ]]; then
       setfattr -n user.component -v "$pkgname" "$filepath" || true
    fi
  done
done

[LukeShortCloud]

Example of step 2:

        env:
          IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.profile }}
          MAX_LAYERS: 128
        run: |
          sudo ./shared/outformat/image/chunkah-package.sh \
          "$IMAGE:${{ steps.version.outputs.tag }}" \
          $(date -d '${{ github.event.head_commit.timestamp }}' +%s)

#!/bin/bash
set -euo pipefail

IMAGE_REF="$1"
SOURCE_DATE_EPOCH="$2"
MAX_LAYERS="${MAX_LAYERS:-128}"

echo "==> Chunkifying $IMAGE_REF (Max Layers: $MAX_LAYERS) - Date: $SOURCE_DATE_EPOCH"

# Get config from existing image
CONFIG=$(podman inspect "$IMAGE_REF")

# Run chunkah (default 64 layers) and pipe to podman load
# Uses --mount=type=image to expose the source image content to chunkah
# Note: We need --privileged for some podman-in-podman/mount scenarios or just standard access
LOADED=$(podman run --rm \
    --security-opt label=type:unconfined_t \
    --mount=type=image,src="$IMAGE_REF",dst=/chunkah \
    -e "CHUNKAH_CONFIG_STR=$CONFIG" \
    -e "SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH" \
    quay.io/jlebon/chunkah@sha256:9d0cb34737ca390df52be72beff22109e4f97f924e0cc155bfdb7e78410e2fc7 \
    build --prune /sysroot/ --max-layers $MAX_LAYERS \
    --label ostree.commit- --label ostree.final-diffid- | podman load)

echo "$LOADED"

# Parse the loaded image reference
NEW_REF=$(echo "$LOADED" | grep -oP '(?<=Loaded image: ).*' || \
          echo "$LOADED" | grep -oP '(?<=Loaded image\(s\): ).*')

if [ -n "$NEW_REF" ] && [ "$NEW_REF" != "$IMAGE_REF" ]; then
    echo "==> Retagging chunked image to $IMAGE_REF..."
    podman tag "$NEW_REF" "$IMAGE_REF"
fi

https://github.com/frostyard/snosi/blob/09d22407f138307f94b6440fac1d3484a53a5bd3/.github/workflows/build-images.yml#L127
https://github.com/frostyard/snosi/blob/09d22407f138307f94b6440fac1d3484a53a5bd3/shared/outformat/image/chunkah-package.sh

The SOURCE_DATE_EPOCH is basically a seed to make the build reproducible with the same container image and tag.

[LukeShortCloud]

The default number of layers chunkah creates is 64 but it is recommended to use 128 for bootc. 500 is the max but that will use too much processing power and slow down the update process.

https://github.com/coreos/chunkah?tab=readme-ov-file#compatibility-with-bootable-bootc-images

[LukeShortCloud]

Testing this with my container-lukeshortcloud-dev project would be a great place to start.