chore: Update to Ubuntu24.04 (cont #7423) #7769

richiejp · 2025-12-29T11:44:56Z

Testing fixes to #7423

ci(workflows): bump GitHub Actions images to Ubuntu 24.04
ci(workflows): remove CUDA 11.x support from GitHub Actions (incompatible with ubuntu:24.04)
ci(workflows): bump GitHub Actions CUDA support to 12.9
build(docker): bump base image to ubuntu:24.04 and adjust Vulkan SDK/packages
fix(backend): correct context paths for Python backends in workflows, Makefile and Dockerfile
chore(make): disable parallel backend builds to avoid race conditions
chore(make): export CUDA_MAJOR_VERSION and CUDA_MINOR_VERSION for override
chore(make): add backends/faster-whisper and docker-save-faster-whisper targets
build(backend): update backend Dockerfiles to Ubuntu 24.04
chore(backend): add ROCm env vars and default AMDGPU_TARGETS for hipBLAS builds
chore(chatterbox): bump ROCm PyTorch to 2.9.1+rocm6.4 and update index URL; align hipblas requirements
chore: add local-ai-launcher to .gitignore
ci(workflows): fix backends GitHub Actions workflows after rebase
build(docker): use build-time UBUNTU_VERSION variable
chore(docker): remove libquadmath0 from requirements-stage base image
chore(make): add backends/vllm to .NOTPARALLEL to prevent parallel builds
chore(make): remove duplicate docker-build-vllm target
fix(docker): correct CUDA installation steps in backend Dockerfiles
chore(backend): update ROCm to 6.4 and align Python hipblas requirements
ci(workflows): switch GitHub Actions runners to Ubuntu-24.04 for CUDA on arm64 builds
build(docker): update base image and backend Dockerfiles for Ubuntu 24.04 compatibility on arm64
build(backend): increase timeout for uv installs behind slow networks on backend/Dockerfile.python
ci(workflows): switch GitHub Actions runners to Ubuntu-24.04 for vibevoice backend
ci(workflows): fix failing GitHub Actions runners
fix: Allow FROM_SOURCE to be unset

netlify · 2025-12-29T11:45:02Z

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	`6d04e23`
🔍 Latest deploy log	https://app.netlify.com/projects/localai/deploys/695d0646f498680008151af6
😎 Deploy Preview	https://deploy-preview-7769--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

richiejp · 2025-12-29T16:11:37Z

we have a chicken and egg problem here with the intel image, so I have set it to the upstream image to test building in the CI.

mudler · 2025-12-29T18:07:25Z

we have a chicken and egg problem here with the intel image, so I have set it to the upstream image to test building in the CI.

we might also not need it anymore, it really was a workaround as I was having issues with using directly upstream images

richiejp · 2025-12-30T09:49:34Z

Cool, OK, all images managed to build (the current build is blocked by a 503 internal server error). I'm guessing the GGML based backends will all be fine at runtime based on previous testing and the Python ones... maybe less so. Do you want to merge this then scramble to fix the resulting issues? @mudler

c.c. @toalex77

mudler · 2026-01-02T08:10:21Z

Cool, OK, all images managed to build (the current build is blocked by a 503 internal server error). I'm guessing the GGML based backends will all be fine at runtime based on previous testing and the Python ones... maybe less so. Do you want to merge this then scramble to fix the resulting issues? @mudler

c.c. @toalex77

yup let's pick it up from master and fix remaining issues there

backend/python/bark/requirements-hipblas.txt

mudler · 2026-01-02T08:11:39Z

backend/Dockerfile.golang

-            else
-                curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu${UBUNTU_VERSION}/arm64/cuda-keyring_1.1-1_all.deb
-            fi
+            curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu${UBUNTU_VERSION}/sbsa/cuda-keyring_1.1-1_all.deb


any specific reason for this? it looks like a regression

I changed this part because this repository https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/arm64/ for Ubuntu 24.04, doesn't contain any of the packages required for the subsequent apt-get installation, but they are available in the sbsa repository.
I then tested the build, and it worked.
Obviously, not having the necessary hardware, I couldn't test the actual functionality of what had been compiled.

Ok, I have both arches (DGX Spark and AGX Orin) so I will be able to test here

mudler · 2026-01-02T08:12:18Z

backend/Dockerfile.golang

            libcublas-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} \
            libcusparse-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} \
            libcusolver-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}
-        if [ "${CUDA_MAJOR_VERSION}" = "13" ] && [ "arm64" = "$TARGETARCH" ]; then


do this work for cuda12?

It appears that these packages exist with CUDA 12. However I am running arm64 locally in QEMU and the build fails later on with what appears to be an error where the x86_64 protoc exe has found its way into the build.

mudler · 2026-01-02T08:12:28Z

backend/Dockerfile.llama-cpp

            curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu${UBUNTU_VERSION}/x86_64/cuda-keyring_1.1-1_all.deb
        fi
        if [ "arm64" = "$TARGETARCH" ]; then
-            if [ "${CUDA_MAJOR_VERSION}" = "13" ]; then


mudler · 2026-01-02T08:12:43Z

backend/Dockerfile.llama-cpp

        apt-get install -y  --no-install-recommends \
            software-properties-common pciutils
        if [ "amd64" = "$TARGETARCH" ]; then
+            echo https://developer.download.nvidia.com/compute/cuda/repos/ubuntu${UBUNTU_VERSION}/x86_64/cuda-keyring_1.1-1_all.deb


this seems an oversight

mudler · 2026-01-02T08:12:52Z

backend/Dockerfile.llama-cpp

            libcublas-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} \
            libcusparse-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} \
            libcusolver-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}
-        if [ "${CUDA_MAJOR_VERSION}" = "13" ] && [ "arm64" = "$TARGETARCH" ]; then


mudler · 2026-01-02T08:13:00Z

backend/Dockerfile.python

-            else
-                curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu${UBUNTU_VERSION}/arm64/cuda-keyring_1.1-1_all.deb
-            fi
+            curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu${UBUNTU_VERSION}/sbsa/cuda-keyring_1.1-1_all.deb


mudler · 2026-01-02T08:13:06Z

backend/Dockerfile.python

            libcusparse-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} \
            libcusolver-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}
-        if [ "${CUDA_MAJOR_VERSION}" = "13" ] && [ "arm64" = "$TARGETARCH" ]; then
+        if [ "arm64" = "$TARGETARCH" ]; then


mudler · 2026-01-02T08:13:35Z

Dockerfile

            libcusparse-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} \
            libcusolver-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}
-        if [ "${CUDA_MAJOR_VERSION}" = "13" ] && [ "arm64" = "$TARGETARCH" ]; then
+        if [ "arm64" = "$TARGETARCH" ]; then


mudler · 2026-01-02T08:13:56Z

Dockerfile

 RUN wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \
 gpg --yes --dearmor --output /usr/share/keyrings/intel-graphics.gpg
-RUN echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy/lts/2350 unified" > /etc/apt/sources.list.d/intel-graphics.list
+RUN echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu noble/lts/2350 unified" > /etc/apt/sources.list.d/intel-graphics.list


good catch, this likely would be better to have its own ARG

mudler · 2026-01-02T08:16:24Z

backend/Dockerfile.python

-COPY python/${BACKEND} /${BACKEND}
-COPY backend.proto /${BACKEND}/backend.proto
-COPY python/common/ /${BACKEND}/common
+COPY backend/python/${BACKEND} /${BACKEND}


mh. this is interesting, some of the Makefile commands where not correctly setting the context path as backend, but here we assume the context now is the whole repository. I tried to avoid this because the main path of LocalAI could likely be in a more dirty state when developing (e.g. model files, etc that then gets dumped in the build context and makes thing slow). any reason to use the . as build context? can we keep it scoped to the backend directory instead?

I think we have to or improve the .dockerignore because I seem to be sending 6GB+ to the context during a backend build.

Hi, I made this change in this commit 9323392 essentially because during the local build tests the Python backends all failed and because, being a different behavior compared to the other backends, I had interpreted it as something to be standardized, but in reality without particular awareness.
So it might be something that needs to be undone and fixed in another way (maybe with some comments explaining why the context is different?).

@toalex77 I see, yes totally should be standardized in one way or another, I tend to prefer isolating the context as much as we can if possible as it hints which files are actually needed (we need only things from the backend directory because there is a common backend.proto, but we don't need anything from the top-level repository)

mudler · 2026-01-02T14:54:48Z

.github/workflows/backend.yml

        include:
-          # CUDA 11 builds
-          - build-type: 'cublas'
-            cuda-major-version: "11"


if we drop cuda 11 (which is totally fine) we should also update docs accordingly

Seems like there was a lot more left over than just the docs, so I have removed everything related to CUDA 11. I believe this will remove support for Kepler GPUs (released 2012), although they may still work with Vulkan for GGML based backends.

richiejp · 2026-01-05T15:22:37Z

"HTTP status server error (503 Service Temporarily Unavailable) for url
#22 67.94 (https://pypi.jetson-ai-lab.io/root/pypi/+f/63c/f8bbe7522de3b/h11-0.16.0-py3-none-any.whl)
" <-- failed twice in a row now.

EDIT: 3 times now, this time with sentencepiece package.

Signed-off-by: Alessandro Sturniolo <[email protected]>

…ible with ubuntu:24.04) Signed-off-by: Alessandro Sturniolo <[email protected]>

…packages Signed-off-by: Alessandro Sturniolo <[email protected]>

… Makefile and Dockerfile Signed-off-by: Alessandro Sturniolo <[email protected]>

Signed-off-by: Alessandro Sturniolo <[email protected]>

…rride Signed-off-by: Alessandro Sturniolo <[email protected]>

Signed-off-by: Alessandro Sturniolo <[email protected]>

…LAS builds Signed-off-by: Alessandro Sturniolo <[email protected]>

…x URL; align hipblas requirements Signed-off-by: Alessandro Sturniolo <[email protected]>

Signed-off-by: Alessandro Sturniolo <[email protected]>

…ilds Signed-off-by: Alessandro Sturniolo <[email protected]>

Signed-off-by: Alessandro Sturniolo <[email protected]>

… on arm64 builds Signed-off-by: Alessandro Sturniolo <[email protected]>

…4.04 compatibility on arm64 Signed-off-by: Alessandro Sturniolo <[email protected]>

… on backend/Dockerfile.python Signed-off-by: Alessandro Sturniolo <[email protected]>

…voice backend Signed-off-by: Alessandro Sturniolo <[email protected]>

Signed-off-by: Alessandro Sturniolo <[email protected]>

Signed-off-by: Richard Palethorpe <[email protected]>

mudler · 2026-01-06T14:26:08Z

let's test on master! it's much easier to catch-up from there given our build matrix. I can also help a bit more to test also on both l4t platforms

github-actions bot added the dependencies label Dec 29, 2025

richiejp mentioned this pull request Dec 29, 2025

build: upgrade base images to Ubuntu 24.04 #7423

Closed

9 tasks

richiejp force-pushed the chore/ubuntu24.04 branch from cc17a26 to e9741ed Compare December 30, 2025 09:38

richiejp marked this pull request as ready for review December 30, 2025 09:39

richiejp force-pushed the chore/ubuntu24.04 branch from e9741ed to 994a8e7 Compare December 30, 2025 09:41

richiejp enabled auto-merge (squash) December 30, 2025 09:42

mudler mentioned this pull request Dec 31, 2025

chore(vulkan): enable arm64 image builds #5780

Closed

1 task

mudler reviewed Jan 2, 2026

View reviewed changes

backend/python/bark/requirements-hipblas.txt Show resolved Hide resolved

mudler reviewed Jan 2, 2026

View reviewed changes

mudler mentioned this pull request Jan 4, 2026

chore(Makefile): refactor common make targets #7858

Merged

1 task

richiejp force-pushed the chore/ubuntu24.04 branch from 9385fb1 to 92d8ce8 Compare January 5, 2026 10:05

github-actions bot added the kind/documentation Improvements or additions to documentation label Jan 5, 2026

toalex77 added 2 commits January 6, 2026 09:22

ci(workflows): bump GitHub Actions images to Ubuntu 24.04

c626fbc

Signed-off-by: Alessandro Sturniolo <[email protected]>

ci(workflows): remove CUDA 11.x support from GitHub Actions (incompat…

168bd8a

…ible with ubuntu:24.04) Signed-off-by: Alessandro Sturniolo <[email protected]>

toalex77 and others added 21 commits January 6, 2026 09:40

build(docker): bump base image to ubuntu:24.04 and adjust Vulkan SDK/…

813f0e6

…packages Signed-off-by: Alessandro Sturniolo <[email protected]>

fix(backend): correct context paths for Python backends in workflows,…

4bc3d8b

… Makefile and Dockerfile Signed-off-by: Alessandro Sturniolo <[email protected]>

chore(make): disable parallel backend builds to avoid race conditions

54cee8e

Signed-off-by: Alessandro Sturniolo <[email protected]>

chore(make): export CUDA_MAJOR_VERSION and CUDA_MINOR_VERSION for ove…

4f02f06

…rride Signed-off-by: Alessandro Sturniolo <[email protected]>

build(backend): update backend Dockerfiles to Ubuntu 24.04

2d41ac3

Signed-off-by: Alessandro Sturniolo <[email protected]>

chore(backend): add ROCm env vars and default AMDGPU_TARGETS for hipB…

ccf588c

…LAS builds Signed-off-by: Alessandro Sturniolo <[email protected]>

chore(chatterbox): bump ROCm PyTorch to 2.9.1+rocm6.4 and update inde…

4850ea3

…x URL; align hipblas requirements Signed-off-by: Alessandro Sturniolo <[email protected]>

chore: add local-ai-launcher to .gitignore

52403f7

Signed-off-by: Alessandro Sturniolo <[email protected]>

ci(workflows): fix backends GitHub Actions workflows after rebase

8c83954

Signed-off-by: Alessandro Sturniolo <[email protected]>

build(docker): use build-time UBUNTU_VERSION variable

b0347d3

Signed-off-by: Alessandro Sturniolo <[email protected]>

chore(docker): remove libquadmath0 from requirements-stage base image

6a84969

Signed-off-by: Alessandro Sturniolo <[email protected]>

chore(make): add backends/vllm to .NOTPARALLEL to prevent parallel bu…

72e4635

…ilds Signed-off-by: Alessandro Sturniolo <[email protected]>

fix(docker): correct CUDA installation steps in backend Dockerfiles

b8b4994

Signed-off-by: Alessandro Sturniolo <[email protected]>

chore(backend): update ROCm to 6.4 and align Python hipblas requirements

b681c3d

Signed-off-by: Alessandro Sturniolo <[email protected]>

ci(workflows): switch GitHub Actions runners to Ubuntu-24.04 for CUDA…

05c9836

… on arm64 builds Signed-off-by: Alessandro Sturniolo <[email protected]>

build(docker): update base image and backend Dockerfiles for Ubuntu 2…

5254fdd

…4.04 compatibility on arm64 Signed-off-by: Alessandro Sturniolo <[email protected]>

build(backend): increase timeout for uv installs behind slow networks…

740a3bb

… on backend/Dockerfile.python Signed-off-by: Alessandro Sturniolo <[email protected]>

ci(workflows): switch GitHub Actions runners to Ubuntu-24.04 for vibe…

cf4c488

…voice backend Signed-off-by: Alessandro Sturniolo <[email protected]>

ci(workflows): fix failing GitHub Actions runners

544c51a

Signed-off-by: Alessandro Sturniolo <[email protected]>

fix: Allow FROM_SOURCE to be unset, use upstream Intel images etc.

0710001

Signed-off-by: Richard Palethorpe <[email protected]>

chore(build): rm all traces of CUDA 11

4198530

Signed-off-by: Richard Palethorpe <[email protected]>

richiejp force-pushed the chore/ubuntu24.04 branch from 3d0d4a4 to 4198530 Compare January 6, 2026 09:40

chore(build): Add Ubuntu codename as an argument

6d04e23

Signed-off-by: Richard Palethorpe <[email protected]>

mudler approved these changes Jan 6, 2026

View reviewed changes

mudler disabled auto-merge January 6, 2026 14:26

mudler merged commit e6ba26c into mudler:master Jan 6, 2026
36 of 100 checks passed

This was referenced Jan 8, 2026

chore(ci): use latest jetpack image for l4t #7926

Merged

chore(ci): roll back l4t-cuda12 configurations #7935

Merged

chore(l4t-12): do not use python 3.12 (wheels are only for 3.10) #7928

Merged

Uh oh!

chore: Update to Ubuntu24.04 (cont #7423) #7769

chore: Update to Ubuntu24.04 (cont #7423) #7769

Uh oh!

Conversation

richiejp commented Dec 29, 2025

Uh oh!

netlify bot commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for localai ready!

Uh oh!

richiejp commented Dec 29, 2025

Uh oh!

mudler commented Dec 29, 2025

Uh oh!

richiejp commented Dec 30, 2025

Uh oh!

mudler commented Jan 2, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

richiejp commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mudler commented Jan 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

netlify bot commented Dec 29, 2025 •

edited

Loading

richiejp commented Jan 5, 2026 •

edited

Loading