Skip to content

feat(pypi): add native local wheel override support to bzlmod pip.parse#3768

Open
meteorcloudy wants to merge 6 commits into
bazel-contrib:mainfrom
meteorcloudy:local-wheel-override
Open

feat(pypi): add native local wheel override support to bzlmod pip.parse#3768
meteorcloudy wants to merge 6 commits into
bazel-contrib:mainfrom
meteorcloudy:local-wheel-override

Conversation

@meteorcloudy
Copy link
Copy Markdown
Contributor

@meteorcloudy meteorcloudy commented May 11, 2026

This feature allows developers to easily test locally built Python wheels (e.g., wheels built locally before running integration tests) without needing to publish them to an index or manually edit requirements_lock.txt.

This is a proper implementation of the local_wheel_dist_folder feature used in JAX/TF/XLA.

How to Use

In your MODULE.bazel file, configure pip.parse with the new local_wheels dictionary mapping:

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")

pip.parse(
    hub_name = "pip",
    python_version = "3.15",
    requirements_lock = "//:requirements_lock.txt",
    experimental_index_url = "https://pypi.org/simple",
    local_wheels = {
        "my_package": "dist/my_package-0.1.0-py3-none-any.whl",
        "libtpu": "dist/libtpu-*.whl",
    },
)

Details

  • Maps individual package names to local wheel paths relative to your workspace root. Supports middle wildcards (e.g. dist/libtpu-*.whl) and automatically selects the newest version if multiple matching wheels exist. Silently falls back to remote PyPI if the local file is missing.
  • Discovery the workspace root via @@//:MODULE.bazel
  • Effective only when pip.parse is evaluated by the root module.
  • Effective only when index_url is used (which is now by default)

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for overriding PyPI dependencies with local wheels located in a specified distribution folder, restricted to the root module. It adds new attributes to the pip extension for specifying the distribution folder and inclusion/exclusion patterns, implements logic to collect and filter local wheels based on ABI markers, and updates the wheel repository generation to use local file URLs. Feedback from the review highlights potential issues with ABI marker generation when patch versions are present and warns against using lexicographical sorting to resolve multiple matching wheels, suggesting instead to fail on ambiguity.

Comment thread python/private/pypi/hub_builder.bzl Outdated
Comment thread python/private/pypi/hub_builder.bzl Outdated
@meteorcloudy meteorcloudy marked this pull request as draft May 11, 2026 15:09
@meteorcloudy meteorcloudy force-pushed the local-wheel-override branch 4 times, most recently from 30e1a50 to 7083382 Compare May 12, 2026 12:48
@meteorcloudy meteorcloudy force-pushed the local-wheel-override branch from 7083382 to ace01f7 Compare May 12, 2026 13:48
@meteorcloudy meteorcloudy force-pushed the local-wheel-override branch from d8b2229 to 8310448 Compare May 12, 2026 13:54
@meteorcloudy meteorcloudy marked this pull request as ready for review May 12, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant