Support building the SDK for arm64

[chewi]

Support building the SDK for arm64

Catalyst 4 adds support for building with QEMU, so I initially leveraged this, but it turned out to be very much slower than emulating the amd64 SDK on arm64, where an arm64 build could then be mostly run without emulation. I have kept the code for the slower approach anyway since it is small and may be useful to somebody.

There were several places where we assumed that amd64 was native and arm64 required emulation via QEMU. The scripts are now more architecture-agnostic, paving the way for riscv support later.

We no longer set QEMU_LD_PREFIX because it prevents the SDK itself from being emulated. It also assumes there is only one non-native target, which may not always be the case. bubblewrap does a better job of running binaries under QEMU. I've put a lot of work into catching all the instances where we were implicitly relying on QEMU when building packages. The new upstream sysroot.eclass has addressed some of the more stubborn cases.

Note that this doesn't address the issue of how we actually publish a supported arm64 SDK on an ongoing basis as that requires more work. It would make sense to have a multi-arch image, but that might require building them in tandem. We wouldn't want a build of one to break the other. The naming of the SDK is also ambiguous in places.

How to use

• Start with Docker on some arm64 distro (e.g. Ubuntu 24.04) running natively.
• Download the SDK .tar.zst tarball.
• Download the latest Gentoo stage3 arm64 systemd tarball.
• Add --exclude dev-lang/perl to the list of excludes in build_library/catalyst.sh. This is a workaround for Gentoo's stage3 being unstable. Maybe we could add this permanently.

docker run --privileged --rm tonistiigi/binfmt --install amd64
zstdcat flatcar-sdk-all-*.tar.zst | docker load
./run_sdk_container -t -U -n amd64-sdk -C ghcr.io/flatcar/flatcar-sdk-all:*
sudo env ARCH=arm64 ./bootstrap_sdk --seed_tarball stage3-arm64-systemd-*.tar.xz --profile coreos-overlay:coreos/arm64/sdk

At this point, my amd64-sdk container had seemingly corrupted itself because it refused to execute amd64 binaries. I don't recall this happening on my earlier attempts, but recreating the container works. The next step is done outside the container anyway.

./build_sdk_container_image __build__/images/catalyst/builds/flatcar-sdk/flatcar-sdk-arm64-*.tar.bz2
./run_sdk_container -t -U -n arm64-sdk -C ghcr.io/flatcar/flatcar-sdk-all:*
./build_packages --board arm64-usr --nousepkg
./build_image --board=arm64-usr prod sysext oem_sysext

I tried also building an amd64 image, and it almost worked, but there a couple of small issues to deal with.

Testing done

This two-phase SDK build in Jenkins was entirely successful. I noted one additional empty /usr/bin/qemu-aarch64 file in the arm64 image, but I've since added a line to clean that up at the end.

I have also built an arm64 SDK as above and used it to build an arm64 image.

• Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update) -- N/A
• Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.