The NVIDIA driver on your system is too old (found version 12080)

[Jonathan-Wei]

Notice: In order to resolve issues efficiently, please follow the template.
（注意：为了更加高效率解决您遇到的问题，请按照模板提问，补充细节。）

Before asking

1. Search existing issues: https://github.com/modelscope/FunASR/issues
2. Search the docs: https://modelscope.github.io/FunASR/
3. Check the README quick start and deployment section.

Question

按照https://github.com/modelscope/FunASR/blob/main/docs/vllm_guide_zh_v2.md，安装后执行python demo_vllm.py --input audio.wav --hotwords 张三 北京 ，提示cuda版本有问题。
安装命令：

pip install torch torchaudio
pip install funasr>=1.3.0
pip install vllm>=0.12.0
pip install safetensors tiktoken websockets regex fastapi uvicorn python-multipart
pip install -e .

cuda

(funasr_vllm) feixin@feixin:/data/deploy/FunASR-new/FunASR/examples/industrial_data_pretraining/fun_asr_nano$ nvidia-smi
Thu Jun 11 18:48:11 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.10              Driver Version: 570.86.10      CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4060 Ti     Off |   00000000:04:00.0 Off |                  N/A |
| 33%   52C    P0             72W /  165W |    7879MiB /  16380MiB |     44%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A          100505      C   python                                  118MiB |
|    0   N/A  N/A         1592328      C   python                                 3178MiB |
|    0   N/A  N/A         1592744      C   VLLM::EngineCore                       4438MiB |
|    0   N/A  N/A         2259378      C   python                                  118MiB |
+-----------------------------------------------------------------------------------------+

Code or command

(funasr_vllm) feixin@feixin:/data/deploy/FunASR-new/FunASR/examples/industrial_data_pretraining/fun_asr_nano$ python demo_vllm.py --input audio.wav --hotwords 张三 北京
============================================================
Fun-ASR-Nano vLLM Inference
============================================================
  Model: FunAudioLLM/Fun-ASR-Nano-2512
  Tensor Parallel: 1 GPU(s)
  Dtype: bf16
  Language: 中文
  Hotwords: ['张三', '北京']

Downloading Model from https://www.modelscope.cn to directory: /home/feixin/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512
Traceback (most recent call last):
  File "/data/deploy/FunASR-new/FunASR/examples/industrial_data_pretraining/fun_asr_nano/demo_vllm.py", line 171, in <module>
    main()
  File "/data/deploy/FunASR-new/FunASR/examples/industrial_data_pretraining/fun_asr_nano/demo_vllm.py", line 63, in main
    engine = FunASRNanoVLLM.from_pretrained(
  File "/data/deploy/FunASR-new/FunASR/funasr/models/fun_asr_nano/inference_vllm.py", line 710, in from_pretrained
    return cls(
  File "/data/deploy/FunASR-new/FunASR/funasr/models/fun_asr_nano/inference_vllm.py", line 184, in __init__
    self._load_audio_components(model_dir, **kwargs)
  File "/data/deploy/FunASR-new/FunASR/funasr/models/fun_asr_nano/inference_vllm.py", line 355, in _load_audio_components
    self.audio_encoder = self.audio_encoder.to(self.device, dtype=torch.float32)
  File "/opt/miniconda/envs/funasr_vllm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1384, in to
    return self._apply(convert)
  File "/opt/miniconda/envs/funasr_vllm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 934, in _apply
    module._apply(fn)
  File "/opt/miniconda/envs/funasr_vllm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 934, in _apply
    module._apply(fn)
  File "/opt/miniconda/envs/funasr_vllm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 934, in _apply
    module._apply(fn)
  [Previous line repeated 1 more time]
  File "/opt/miniconda/envs/funasr_vllm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 965, in _apply
    param_applied = fn(param)
  File "/opt/miniconda/envs/funasr_vllm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1370, in convert
    return t.to(
  File "/opt/miniconda/envs/funasr_vllm/lib/python3.10/site-packages/torch/cuda/__init__.py", line 478, in _lazy_init
    torch._C._cuda_init()
RuntimeError: The NVIDIA driver on your system is too old (found version 12080). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver.

What have you tried?

Environment

• OS: Ubuntu24.04
• Python version: 13.10
• FunASR version: 1.3.9
• ModelScope version: 1.37.1
• PyTorch / torchaudio version: 2.11.0
• Install method (pip, source, Docker):
• Device (cuda, cpu, mps): cuda
• GPU model:
• CUDA/cuDNN version: 12.8
• Docker image tag, if used:

[LauraGPT]

这个报错发生在 torch._C._cuda_init()，也就是 PyTorch CUDA 初始化阶段，还没进入 FunASR/vLLM 推理逻辑。根因通常是当前安装的 PyTorch/vLLM wheel 依赖的 CUDA runtime 高于 NVIDIA driver 能支持的范围。

你的机器显示：

• Driver: 570.86.10
• nvidia-smi CUDA: 12.8
• PyTorch: 2.11.0

建议先确认当前 wheel 实际编译的 CUDA 版本：

python - <<'PY'
import torch
print("torch", torch.__version__, "cuda", torch.version.cuda)
try:
    import vllm
    print("vllm", vllm.__version__)
except Exception as e:
    print("vllm import failed:", e)
PY

处理方向：

1. 不要无约束执行 pip install torch torchaudio，这可能拉到与你驱动不匹配的最新 CUDA wheel。
2. 重新安装与 nvidia-smi/driver 匹配的 PyTorch、torchaudio、vLLM 组合，或先升级 NVIDIA driver。
3. 重装后先跑下面的最小 CUDA 检查，再跑 demo_vllm.py：

python - <<'PY'
import torch
print(torch.__version__, torch.version.cuda)
print(torch.cuda.is_available())
print(torch.cuda.get_device_name(0))
PY

我也已在 vLLM 文档里补了这个注意事项，避免继续被 pip install vllm>=0.12.0 误导：modelscope/FunASR@ef46055

[qiulang]

@LauraGPT 我再提点意见 （因为最近一直在看vllm + Fun-ASR-Nano-2512 所以挺有兴趣参加这样的讨论）

他这个错其实挺容易反推出原因： "The NVIDIA driver on your system is too old Please update your GPU " 这就很容易反推他下的torch是基于 cuda13编译的，而他目前的cuda是12 （nvidia-smi显示的），所以自然就报错。

发生的原因应该也容易推理出 pip install torch 没指定版本

curl -s https://pypi.org/pypi/torch/2.11.0/json | grep -o 'nvidia-[a-z]*-cu1[0-9]'

nvidia-cudnn-cu13
nvidia-cusparselt-cu13
nvidia-nccl-cu13
nvidia-nvshmem-cu13

安装torch确实是一个大坑，我被坑了好几次，他这个版本要解决就是

pip install torch==2.11.0 torchaudio --index-url https://download.pytorch.org/whl/cu128

另外可以参看这个对照表 https://github.com/pytorch/pytorch/blob/main/RELEASE.md#pytorch-cuda-support-matrix

看看装哪个版本最合适当前系统

另外我现在解决这个问题办法就是不执行 pip install torch 而是只执行 pip install vllm 但要根据cuda指定vllm版本

if [[ "$CUDA_MAJOR" -ge 13 ]]; then
    echo "=== CUDA >= 13: installing latest vLLM (torch cu130) ==="
    uv pip install vllm
else
    echo "=== CUDA < 13 (12.x driver): installing vLLM 0.19.1 (torch cu128) ==="
    uv pip install vllm==0.19.1
fi

因为 vllm会指定它需要的 torch版本 https://github.com/vllm-project/vllm/blob/main/pyproject.toml

[LauraGPT]

@qiulang 这个补充很有价值，我同意这里不能简单写成“装某个固定版本”，核心应该按 NVIDIA driver 可支持的 CUDA runtime / PyTorch wheel / vLLM wheel 三者一起匹配。

这个 issue 里的报错确实可以从 driver too old 反推出当前环境大概率装到了依赖更高 CUDA runtime 的 torch/vLLM wheel。更稳的排查顺序我建议是：

1. 先看机器 driver 能支持到哪个 CUDA runtime：nvidia-smi 里的 CUDA Version。
2. 再看当前 Python 环境实际装的 wheel：

python - <<PY
import torch
print("torch", torch.__version__)
print("torch cuda", torch.version.cuda)
try:
    import vllm
    print("vllm", vllm.__version__)
except Exception as e:
    print("vllm import failed:", repr(e))
PY

3. 避免无约束执行 pip install torch torchaudio 或 pip install -U ...，否则很容易拉到与 driver 不匹配的 CUDA wheel。
4. 如果要显式安装 torch，就用 PyTorch 官方对应 CUDA index，例如 cu128 / cu126 这种明确 index；如果主要跑 vLLM，也可以像你说的让 vLLM 依赖约束带出匹配的 torch，但仍然要按本机 driver/CUDA runtime 选择 vLLM 版本。

我后面会把文档里的安装建议继续往“按 driver/CUDA 组合选择”这个方向收敛，避免给用户一个看起来通用、实际容易踩坑的固定版本命令。