[BUG]: cuda-core 1.2.0 breaks Buffer use for non-CUDA memory implementations

[carterbox]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this bug and that I agree to the Code of Conduct

Type of Bug

Runtime Error

Component

cuda.core

Describe the bug

nvmath-python provides NVPL-backed host APIs; this means that we run in environments where the CUDA driver is not present or the context is not initialized.

I have implemented a MemoryResource sub-class that uses Numpy as the allocator. This is because I want to use the same interfaces for both CUDA and Non-CUDA when performing internal memory allocations.

Starting in cuda-core 1.2.0, it is now not possible to inherit from MemoryResource/Buffer without running cuInit? For example, I get the following error when running tests that do not use CUDA.

nvmath/_internal/workspace.py:519: in allocate
    return Buffer.from_handle(ptr=ptr, size=size, mr=self)
cuda/core/_memory/_buffer.pyx:379: in cuda.core._memory._buffer.Buffer.from_handle
    ???
cuda/core/_memory/_buffer.pyx:310: in cuda.core._memory._buffer.Buffer._init
    ???
cuda/core/_utils/cuda_utils.pxd:23: in cuda.core._utils.cuda_utils.HANDLE_RETURN
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   cuda.core._utils.cuda_utils.CUDAError: CUDA_ERROR_NOT_INITIALIZED: This indicates that the CUDA driver has not been initialized with cuInit() or that initialization has failed.

How to Reproduce

import numpy as np

from cuda.core import MemoryResource, Buffer, Stream


class NumpyMemoryResource(MemoryResource):
    """Host :class:`cuda.core.MemoryResource` backed by :func:`numpy.empty`.

    Allocates plain (non-pinned) host memory via NumPy. Unlike
    :class:`PinnedMemoryResource` / :class:`LegacyPinnedMemoryResource`, this
    resource does not call into the CUDA driver, so it is safe to use in
    environments where no CUDA driver/device is available (CPU-only test runs,
    pure-host scratch space for parameter struct staging, etc.).

    Pair with ``Workspace(NumpyMemoryResource(), logger)`` and pass
    ``stream_holder=None`` to ``allocate_perhaps`` for a fully CUDA-free path
    (``device_id`` is read from the resource and is ``-1``).

    Not suitable as the host half of a :class:`MirroredWorkspace`: ``cudaMemcpyAsync``
    requires page-locked memory for asynchronous correctness.
    """

    def __init__(self) -> None:
        # Strong refs keyed by ptr — Buffer carries only the int pointer.
        self._held: dict[int, np.ndarray] = {}

    def allocate(self, size: int, *, stream: Stream | None = None) -> Buffer:
        arr = np.empty(size, dtype=np.uint8)
        ptr = int(arr.ctypes.data)
        self._held[ptr] = arr
        return Buffer.from_handle(ptr=ptr, size=size, mr=self)

    def deallocate(self, ptr: int, size: int, stream: Stream | None = None) -> None:
        # `stream` must NOT be keyword-only: cuda.core < 1 calls deallocate with all
        # arguments positionally (Buffer.close -> mr.deallocate(ptr, size, stream)).
        self._held.pop(int(ptr), None)

    @property
    def is_host_accessible(self) -> bool:
        return True

    @property
    def is_device_accessible(self) -> bool:
        return False

    @property
    def is_managed(self) -> bool:
        return False

    @property
    def device_id(self) -> int:
        return -1

if __name__ == "__main__":
    mr = NumpyMemoryResource()
    b = mr.allocate(64, stream=None)

Expected behavior

No runtime error from Buffer constructor if the CUDA runtime hasn't been initialized.

Operating System

WSL2 Ubuntu 24.04

nvidia-smi output

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.79                 Driver Version: 596.59         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX 1000 Ada Gene...    On  |   00000000:01:00.0 Off |                  N/A |
| N/A   52C    P3             11W /   44W |       0MiB /   6141MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

[Andy-Jost]

Thanks for the detailed report and the reproducer.

Root cause

Since 1.2.0, cuda.core records a deallocation stream when a Buffer is created with mr= (#2526). When no stream= is given, the default stream token is used. That token carries no context, so Buffer.from_handle looks up the current context with cuCtxGetCurrent and pins it. This lets the later free run in the allocation context even when the buffer is released from another thread or after the current context changed (#2497).

Two things go wrong in your scenario:

1. The lookup runs for every owning Buffer, including buffers from a memory resource that never touches CUDA. Memory that the device cannot access is not a CUDA allocation, and freeing it needs no context, so the lookup is unnecessary there.
2. When the driver has not been initialized, the lookup fails with a different error code than the one from_handle translates into the descriptive RuntimeError ("no CUDA context is current ..."), so the raw CUDAError escapes instead.

The documented remedies in 1.2.0 (Device.set_current() or a non-default stream=) both require an initialized driver, so a CUDA-free process has no way to create an owning Buffer today. That is a regression from 1.1.x.

Fix

When mr.is_device_accessible is False, Buffer.from_handle records no deallocation stream and does not call the driver. mr.deallocate() still receives the default stream, as in 1.1. Device-accessible resources are unchanged. The change is in #2773, with tests that run a NumPy-backed resource like yours in a process that never initializes CUDA.

Workaround for 1.2.0

Pass owner=arr instead of mr=self; the array's refcount frees the memory. buf.handle and buf.size work, but is_host_accessible, is_device_accessible, and device_id query the driver when no memory resource is attached, so avoid them in CUDA-free code. Alternatively, pin cuda-core<1.2.