faiss-cpu: add build-faiss-cpu.yml for riscv64 wheels (#530)

* faiss-cpu: add build-faiss-cpu.yml for riscv64 wheels

faiss v1.15.0 already carries riscv64 RVV kernels and builds its wheels
with plain cibuildwheel driven entirely from pyproject.toml, so the
workflow is upstream's build-pip.yml narrowed to one riscv64 job.

Two patches are needed on top:

  - the RVV sources are compiled with -march=rv64gcv_zvfhmin, which
    binutils 2.41 in manylinux_2_39_riscv64 (Rocky Linux 10) rejects
    outright; none of them uses float16 vectors, so probe the flag and
    fall back to rv64gcv;

  - SIMDConfig::detect() offers SIMDLevel::RISCV_RVV unconditionally
    while only the RVV kernels are built with -march=rv64gcv, so the
    wheel would SIGILL on the rv64gc baseline the platform tag promises;
    read AT_HWCAP instead, as the aarch64 SVE path already does.

pyproject.toml sets wheel.py-api = "cp310" on Linux, so one abi3 wheel
covers every non-free-threaded CPython and upstream ships no
free-threaded wheel at all. It is built on cp312 rather than upstream's
cp310 because the registry's newest riscv64 numpy/scipy stop at 2.2.2 and
1.15.2 for cp310 while PyPI offers 2.2.6 and 1.15.3, which pip would
prefer and then build from sdist.

Signed-off-by: Ludovic Henry <git@ludovic.dev>

* faiss-cpu: publish the image's gcc sources alongside the wheels

auditwheel vendors libgomp and libgfortran out of the manylinux image into
faiss_cpu.libs/, so their sources have to be published with the wheel.

Signed-off-by: Ludovic Henry <git@ludovic.dev>

* faiss-cpu: keep the RVV kernels out of LTO

FAISS_USE_LTO is ON for Linux and only the RVV sources carry
-march=rv64gcv, so lto1 regenerates them at link time from the link
command line, which has no vector extension, and libfaiss.so fails with
"target specific builtin not available".  Compile those seven files with
-fno-lto.

Folded into the existing CMake patch rather than added as a third one:
git apply resolves a glob of patches against the pre-patch tree, so a
patch editing a line an earlier patch added does not apply.

Signed-off-by: Ludovic Henry <git@ludovic.dev>

* faiss-cpu: probe RVV with riscv_hwprobe, fall back to AT_HWCAP

riscv_hwprobe(RISCV_HWPROBE_KEY_IMA_EXT_0) is the authoritative source for
the V extension. glibc only declares __riscv_hwprobe() from 2.40 and the
manylinux image is 2.39, so the syscall is issued directly; __NR_riscv_hwprobe
is in the image's kernel headers (258). Kernels before 6.4 do not implement
it and fail with ENOSYS, which falls back to the AT_HWCAP bit as before.

Compile-tested in quay.io/pypa/manylinux_2_39_riscv64 with -Wall -Wextra.

---------

Signed-off-by: Ludovic Henry <git@ludovic.dev>

Author: luhenry

.github/workflows/build-faiss-cpu.yml

@@ -0,0 +1,129 @@
+# SPDX-FileCopyrightText: 2026 The RISE Project
+# SPDX-License-Identifier: MIT
+---
+# This workflow is based on: https://github.com/facebookresearch/faiss/blob/v1.15.0/.github/workflows/build-pip.yml
+name: Build faiss-cpu wheels (riscv64)
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'faiss-cpu version to build (git tag without the leading v, e.g. 1.15.0)'
+        required: true
+        default: '1.15.0'
+  pull_request:
+    paths:
+      - '.github/workflows/build-faiss-cpu.yml'
+      - 'patches/faiss-cpu/**'
+
+concurrency:
+  group: ${{ github.workflow }}-${{ inputs.version || '1.15.0' }}-${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read  # to fetch code (actions/checkout)
+
+env:
+  # `inputs.version` is empty on pull_request events; default to 1.15.0 there.
+  FAISS_CPU_VERSION: ${{ inputs.version || '1.15.0' }}
+  MANYLINUX_RISCV64_IMAGE: quay.io/pypa/manylinux_2_39_riscv64
+
+jobs:
+  # pyproject.toml sets wheel.py-api = "cp310" on Linux, so a single abi3 wheel
+  # covers every non-free-threaded CPython; upstream skips free-threaded builds
+  # entirely (`skip = "... cp3*t-*"`). Built on cp312 rather than upstream's
+  # cp310 because our registry's newest riscv64 numpy/scipy are cp311+ only.
+  build_wheels:
+    name: Build faiss-cpu ${{ inputs.version || '1.15.0' }} cp310-abi3-manylinux_riscv64
+    runs-on: ubuntu-24.04-riscv
+    timeout-minutes: 600
+
+    steps:
+      - name: Checkout faiss v${{ env.FAISS_CPU_VERSION }}
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0  # v7.0.0
+        with:
+          repository: facebookresearch/faiss
+          ref: v${{ env.FAISS_CPU_VERSION }}
+          persist-credentials: false
+
+      - name: Checkout python-wheels
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0  # v7.0.0
+        with:
+          path: python-wheels
+          persist-credentials: false
+
+      - name: Patch faiss source
+        run: git apply python-wheels/patches/faiss-cpu/${{ env.FAISS_CPU_VERSION }}/*.patch
+
+      - name: Build wheels
+        uses: pypa/cibuildwheel@1828c10ab37f080699c7b81cea34097c684a7074  # v4.2.0
+        with:
+          output-dir: wheelhouse/
+          only: cp312-manylinux_riscv64
+        env:
+          CIBW_MANYLINUX_RISCV64_IMAGE: ${{ env.MANYLINUX_RISCV64_IMAGE }}
+          # numpy (build and test), scipy and swig have no riscv64 wheels on PyPI.
+          CIBW_ENVIRONMENT: PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/
+
+      - name: Check the wheel is a compiled abi3 wheel
+        run: |
+          python3 - wheelhouse/*.whl <<'EOF'
+          import sys, zipfile
+          for whl in sys.argv[1:]:
+              assert "-cp310-abi3-" in whl, whl
+              names = zipfile.ZipFile(whl).namelist()
+              for want in ("faiss/libfaiss.so", "faiss/_swigfaiss.abi3.so"):
+                  assert want in names, (whl, want)
+              print(whl, "ok")
+          EOF
+
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a  # v7.0.1
+        with:
+          name: faiss-cpu-${{ env.FAISS_CPU_VERSION }}-cp310-abi3-manylinux_riscv64
+          path: wheelhouse/*.whl
+          if-no-files-found: error
+
+  gpl_sources:
+    name: Collect GPL sources
+    runs-on: ubuntu-24.04-riscv
+
+    steps:
+      - name: Checkout python-wheels
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0  # v7.0.0
+        with:
+          persist-credentials: false
+
+      # faiss links OpenMP and OpenBLAS, so auditwheel vendors the image's
+      # libgomp and libgfortran into faiss_cpu.libs/.
+      - uses: ./actions/collect-gpl-sources
+        with:
+          image: ${{ env.MANYLINUX_RISCV64_IMAGE }}
+          packages: gcc
+          output: gpl-sources.tar
+
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a  # v7.0.1
+        with:
+          name: faiss-cpu-${{ env.FAISS_CPU_VERSION }}-gpl-sources
+          path: gpl-sources.tar
+          if-no-files-found: error
+
+  publish:
+    name: Publish faiss-cpu ${{ inputs.version || '1.15.0' }} to GitLab
+    needs: [build_wheels, gpl_sources]
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pull-requests: write
+
+    steps:
+      - name: Publish wheels and open docs PR
+        uses: riseproject-dev/python-wheels/actions/publish-wheels@main
+        with:
+          artifact-pattern: faiss-cpu-${{ env.FAISS_CPU_VERSION }}-*-manylinux_riscv64
+          gitlab-username: ${{ vars.GITLAB_DEPLOY_USER }}
+          gitlab-token: ${{ secrets.GITLAB_DEPLOY_TOKEN }}
+          gitlab-project-id: ${{ vars.GITLAB_PROJECT_ID }}
+          gh-token: ${{ secrets.GITHUB_TOKEN }}
+          gpl-sources-artifact: faiss-cpu-${{ env.FAISS_CPU_VERSION }}-gpl-sources
+          gpl-sources-release-tag: faiss-cpu-v${{ env.FAISS_CPU_VERSION }}
+          gpl-sources-description: gcc

patches/faiss-cpu/1.15.0/0001-build-make-the-RVV-kernels-build-with-a-distro-toolc.patch

@@ -0,0 +1,83 @@
+From 5d56ba18506d4e1ed92330fcd368902c67bbb884 Mon Sep 17 00:00:00 2001
+From: Ludovic Henry <git@ludovic.dev>
+Date: Thu, 27 Aug 2026 07:16:00 +0200
+Subject: [PATCH 1/2] build: make the RVV kernels build with a distro toolchain
+
+Upstream-Status: To upstream [not yet submitted; needs a maintainer decision on the intended riscv64 toolchain baseline, so it is not a drive-by PR]
+
+Two things stop the FAISS_SIMD_RVV_SRC files from producing a shared
+library with GCC 14.3.1 / binutils 2.41, the toolchain in the riscv64
+manylinux image (manylinux_2_39_riscv64, Rocky Linux 10).
+
+They are compiled with -march=rv64gcv_zvfhmin.  zvfhmin only reached
+binutils in 2.42, and an older assembler rejects the whole -march string
+rather than just that extension, so every RVV translation unit fails with
+"unknown prefixed ISA extension `zvfhmin'".  None of the seven files uses
+float16 vectors, so zvfhmin buys nothing today and all seven compile
+unchanged with plain rv64gcv.  Probe for the flag and fall back.
+
+FAISS_USE_LTO is also ON for Linux, and with -flto those objects hold
+GIMPLE rather than code, so lto1 regenerates them at link time from the
+*link* command line, which carries no vector extension:
+
+    lto1: fatal error: target specific builtin not available
+    lto-wrapper: fatal error: /usr/bin/g++ returned 1 exit status
+
+GCC has no per-function target attribute for RISC-V vector, so the flags
+cannot be carried into the LTO recompilation.  Compile just those seven
+files with -fno-lto; every other translation unit still takes part.
+
+Signed-off-by: Ludovic Henry <git@ludovic.dev>
+---
+ faiss/CMakeLists.txt | 21 +++++++++++++++++++--
+ 1 file changed, 19 insertions(+), 2 deletions(-)
+
+diff --git a/faiss/CMakeLists.txt b/faiss/CMakeLists.txt
+index 1a4a30a..6deb907 100644
+--- a/faiss/CMakeLists.txt
++++ b/faiss/CMakeLists.txt
+@@ -87,6 +87,23 @@ elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "(riscv64|riscv)")
+ else()
+   set(FAISS_SIMD_SRC "")
+ endif()
++
++# zvfhmin is only recognised by binutils >= 2.42. Older assemblers reject the
++# whole -march string, and no RVV source here uses float16 vectors, so probe
++# for it and fall back to plain rv64gcv.
++# The RVV kernels must be excluded from LTO: lto1 recompiles them from GIMPLE
++# using the link command's target options, which do not include the vector
++# extension, and fails with "target specific builtin not available".
++set(FAISS_RVV_ARCH_FLAGS "-march=rv64gcv_zvfhmin;-mabi=lp64d;-fno-lto")
++if(CMAKE_SYSTEM_PROCESSOR MATCHES "(riscv64|riscv)")
++  include(CheckCXXSourceCompiles)
++  set(CMAKE_REQUIRED_FLAGS "-march=rv64gcv_zvfhmin -mabi=lp64d")
++  check_cxx_source_compiles("int main() { return 0; }" FAISS_HAVE_RVV_ZVFHMIN)
++  unset(CMAKE_REQUIRED_FLAGS)
++  if(NOT FAISS_HAVE_RVV_ZVFHMIN)
++    set(FAISS_RVV_ARCH_FLAGS "-march=rv64gcv;-mabi=lp64d;-fno-lto")
++  endif()
++endif()
+ # =============================================================================
+ 
+ set(FAISS_SRC
+@@ -595,7 +612,7 @@ if(FAISS_OPT_LEVEL STREQUAL "dd")
+     if(FAISS_SIMD_RVV_SRC)
+       set_source_files_properties(${FAISS_SIMD_RVV_SRC}
+         TARGET_DIRECTORY faiss
+-        PROPERTIES COMPILE_OPTIONS "-march=rv64gcv_zvfhmin;-mabi=lp64d"
++        PROPERTIES COMPILE_OPTIONS "${FAISS_RVV_ARCH_FLAGS}"
+       )
+     endif()
+   endif()
+@@ -624,7 +641,7 @@ if(CMAKE_SYSTEM_PROCESSOR MATCHES "(riscv64|riscv)" AND NOT FAISS_OPT_LEVEL STRE
+   if(NOT WIN32 AND FAISS_SIMD_RVV_SRC)
+     set_source_files_properties(${FAISS_SIMD_RVV_SRC}
+       TARGET_DIRECTORY faiss
+-      PROPERTIES COMPILE_OPTIONS "-march=rv64gcv_zvfhmin;-mabi=lp64d"
++      PROPERTIES COMPILE_OPTIONS "${FAISS_RVV_ARCH_FLAGS}"
+     )
+   endif()
+ endif()
+-- 
+2.50.1 (Apple Git-155)
+

patches/faiss-cpu/1.15.0/0002-Detect-RVV-at-runtime-instead-of-assuming-it-on-risc.patch

@@ -0,0 +1,100 @@
+From 2ae2e10260375f9aec6d6d546b60f7f9e621b86e Mon Sep 17 00:00:00 2001
+From: Ludovic Henry <git@ludovic.dev>
+Date: Thu, 27 Aug 2026 06:31:39 +0200
+Subject: [PATCH 2/2] Detect RVV at runtime instead of assuming it on riscv64
+
+Upstream-Status: To upstream [not yet submitted; needs a maintainer decision on the intended riscv64 baseline, so it is not a drive-by PR]
+
+Only the files in FAISS_SIMD_RVV_SRC are compiled with -march=rv64gcv; the
+rest of the library targets the toolchain default, which for a distro
+compiler is the rv64gc baseline.  SIMDConfig::detect() nevertheless sets
+SIMDLevel::RISCV_RVV unconditionally, so a build that runs on a CPU
+without the V extension dispatches into vector kernels and dies with
+SIGILL.  Most riscv64 hardware in circulation (JH7110, HiFive P550) has no
+vector unit, and a manylinux_riscv64 wheel is expected to run on the
+rv64gc baseline.
+
+Probe with riscv_hwprobe(RISCV_HWPROBE_KEY_IMA_EXT_0) and offer RISCV_RVV
+only when RISCV_HWPROBE_IMA_V is set.  glibc exposes __riscv_hwprobe() only
+from 2.40 and the manylinux image is 2.39, so the syscall is issued directly.
+Kernels before 6.4 do not implement it and fail with ENOSYS; those fall back
+to the AT_HWCAP bit, mirroring the existing has_sve() aarch64 path.  dispatch_with_fallback() already falls
+back from RISCV_RVV to SIMDLevel::NONE, so the scalar kernels take over on
+hardware that lacks it.
+
+Signed-off-by: Ludovic Henry <git@ludovic.dev>
+---
+ faiss/utils/simd_levels.cpp | 28 +++++++++++++++++++++++++---
+ 1 file changed, 25 insertions(+), 3 deletions(-)
+---
+
+diff --git a/faiss/utils/simd_levels.cpp b/faiss/utils/simd_levels.cpp
+index e17796b..dd6a996 100644
+--- a/faiss/utils/simd_levels.cpp
++++ b/faiss/utils/simd_levels.cpp
+@@ -57,6 +57,45 @@
+ }
+ #endif
+ 
++// RISC-V vector runtime detection
++#if defined(__riscv) && defined(__linux__)
++
++#include <sys/auxv.h>
++#include <sys/syscall.h>
++#include <unistd.h>
++#include <cstdint>
++
++#ifndef COMPAT_HWCAP_ISA_V
++#define COMPAT_HWCAP_ISA_V (1 << ('V' - 'A'))
++#endif
++
++// glibc only declares __riscv_hwprobe() from 2.40 and manylinux_2_39 is older,
++// so issue the syscall directly.  Kernels before 6.4 do not implement it and
++// fail with ENOSYS; AT_HWCAP is the fallback for those.
++#ifndef __NR_riscv_hwprobe
++#define __NR_riscv_hwprobe 258
++#endif
++#define RISCV_HWPROBE_KEY_IMA_EXT_0 4
++#define RISCV_HWPROBE_IMA_V (1ULL << 2)
++
++static bool has_rvv() {
++    struct {
++        int64_t key;
++        uint64_t value;
++    } pair = {RISCV_HWPROBE_KEY_IMA_EXT_0, 0};
++
++    if (syscall(__NR_riscv_hwprobe, &pair, 1UL, 0UL, nullptr, 0U) == 0) {
++        return (pair.value & RISCV_HWPROBE_IMA_V) != 0;
++    }
++    return (getauxval(AT_HWCAP) & COMPAT_HWCAP_ISA_V) != 0;
++}
++
++#else
++[[maybe_unused]] static bool has_rvv() {
++    return false;
++}
++#endif
++
+ // Detect x86 microarchitecture flags used for kernel routing. Uses raw
+ // cpuid so it is safe to run on any CPU regardless of compiled SIMD level.
+ #if defined(__x86_64__)
+@@ -244,9 +283,13 @@ SIMDLevel SIMDConfig::auto_detect_simd_level() {
+ #endif
+ 
+ #if defined(__riscv) && defined(COMPILE_SIMD_RISCV_RVV)
+-    // RVV is always available on RISC-V builds compiled with rv64gcv.
+-    supported_simd_levels |= (1 << static_cast<int>(SIMDLevel::RISCV_RVV));
+-    detected_level = SIMDLevel::RISCV_RVV;
++    // Only the RVV kernels are compiled with -march=rv64gcv; the rest of the
++    // library targets the rv64gc baseline, so they must not be dispatched to
++    // on a CPU without the V extension.
++    if (has_rvv()) {
++        supported_simd_levels |= (1 << static_cast<int>(SIMDLevel::RISCV_RVV));
++        detected_level = SIMDLevel::RISCV_RVV;
++    }
+ #endif
+ 
+     return detected_level;
+-- 
+2.50.1 (Apple Git-155)
+