cuda.core: directory-aware C++ build rule and drop the dead top-level copy from the merged wheel - #2799
Open
Andy-Jost wants to merge 3 commits into
Open
Conversation
github-actions
Bot
added
CI/CD
… copy from the merged wheel build_hooks.py maps a Cython module to its C++ by name. It now also accepts a directory: every .cpp under cuda/core/_cpp/<stem>/ compiles into the one extension for _<stem>.pyx, with the legacy single file _cpp/<stem>.cpp kept as the fallback (tensor_map is unchanged). With no such directory in the tree yet, the sources are exactly today's, so this part is inert on its own. The cuda.core._cpp package-data globs become recursive so headers in nested directories ship, and .gitignore stops ignoring .cpp files under cuda/core/_cpp/ so new sources are visible to git. ci/tools/merge_cuda_core_wheels.py stops retaining a third, top-level copy of _resource_handles and the top-level _cpp/ and _include/ headers in the merged cu12+cu13 wheel. cuda/core/__init__.py rewrites __path__ to the versioned subpackage before any import reaches them, so that copy (about 308 KB uncompressed in the 1.2.0 wheel) was never imported; the comment defending it referred to an import removed in NVIDIA#1463. A step in build-wheel.yml now asserts that the merged wheel's top level holds only __init__.py, _version.py and the two versioned trees. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Andy-Jost
force-pushed
the
ajost/rt-phase1-build-and-wheel-fix
branch
from
September 10, 2026 02:35
2ae8604 to
cb5784e
Compare
Contributor
|
mdboom
requested changes
Sep 10, 2026
Comment on lines
+699
to
+718
| - name: Check merged cuda.core wheel layout | ||
| if: ${{ env.BUILD_CORE == 'true' }} | ||
| run: | | ||
| # cuda/core/__init__.py rewrites __path__ to the active cuXX/ tree, so | ||
| # only the entry points and the two versioned trees belong at the top | ||
| # level of cuda/core/; anything else is a dead copy. | ||
| python - "${{ env.CUDA_CORE_ARTIFACTS_DIR }}"/cuda_core*.whl <<'EOF' | ||
| import os | ||
| import sys | ||
| import zipfile | ||
|
|
||
| prefix = "cuda/core/" | ||
| members = zipfile.ZipFile(sys.argv[1]).namelist() | ||
| names = {n[len(prefix) :].split("/")[0] for n in members if n.startswith(prefix)} | ||
| names.discard("") | ||
| expected = {"__init__.py", "_version.py"} | ||
| expected |= {f"cu{os.environ[v]}" for v in ("BUILD_CUDA_MAJOR", "BUILD_PREV_CUDA_MAJOR")} | ||
| assert names == expected, f"unexpected top-level entries under cuda/core/: {sorted(names ^ expected)}" | ||
| print("merged wheel top level:", sorted(names)) | ||
| EOF |
Contributor
There was a problem hiding this comment.
Since we already have a Python script to merge cuda-core wheels, maybe it's simplest to just add this check there? (This on its own isn't terrible, but writ large we have too much logic in our GHA config.)
Contributor
Author
There was a problem hiding this comment.
Done in 28b73d2: the check now lives in merge_cuda_core_wheels.py, which derives the kept cuNN directories from its input wheels and raises if anything else remains under cuda/core/. The workflow step is gone, so build-wheel.yml is back to what main has.
| cuda/core/_cpp/<stem>/, or the single legacy file cuda/core/_cpp/<stem>.cpp. | ||
| Example: _tensor_map.pyx compiles _cpp/tensor_map.cpp.""" | ||
| sources = [f"cuda/core/{mod_name}.pyx"] | ||
| cpp_stem = os.path.join("cuda", "core", "_cpp", mod_name.lstrip("_")) |
Contributor
There was a problem hiding this comment.
os.path is soft deprecated. We should use pathlib.Path in new code.
Contributor
Author
There was a problem hiding this comment.
Done in 28b73d2: _extension_sources uses pathlib.Path.
…e pathlib Review follow-up. The top-level layout assertion moves from build-wheel.yml into ci/tools/merge_cuda_core_wheels.py, which now derives the kept cuNN directories from its input wheels and raises if anything else remains under cuda/core/ after the removal. _extension_sources uses pathlib.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Groundwork for splitting
cuda.core._resource_handlesinto a directory of C++ sources, plus a fix to the merged-wheel layout. No behavior changes.Build backend.
build_hooks.pynow compiles every.cppundercuda/core/_cpp/<stem>/into the extension for_<stem>.pyx, keeping the legacy single file_cpp/<stem>.cppas the fallback (tensor_mapis unchanged). With no such directory in the tree yet, the sources it produces are exactly today's, so this part is inert on its own. Thecuda.core._cpppackage-data globs become recursive so headers in nested directories ship, and.gitignorestops hiding.cppfiles undercuda/core/_cpp/so new sources are visible to git.Merged wheel.
ci/tools/merge_cuda_core_wheels.pystops keeping a third, top-level copy of_resource_handles(extension,.pxd,.pyi) and the top-level_cpp/and_include/header directories in the merged cu12+cu13 wheel.cuda/core/__init__.pyrewrites the package's__path__,__file__, and__spec__to the versioned subpackage before any import reaches them, so that copy was never used at runtime (about 308 KB uncompressed in the 1.2.0 wheel); the comment defending it referred to an import removed in #1463. The C++ headers advertised in 1.1.1 (#2236) are still shipped undercu12/andcu13/and are reached through the package's import path, which resolves to the active one. After the removal the script asserts that the merged wheel's top level holds exactly__init__.py,_version.py, and the versioned trees of its input wheels.Tests. Three small tests of the source rule in
test_build_hooks.py(directory form, legacy single file, no C++, empty directory).Checklist
🤖 Generated with Claude Code