GPU版BERTopic运行出现cuBLAS初始化错误求助

执行GPU版BERTopic时触发cuBLAS初始化错误，切换为CPU模式后错误消失，推测为依赖版本不兼容问题。

错误日志 #

Traceback (most recent call last):
  File "/mnt/JianFeng/2_Language_Models/2_3_0_BERTopic_Debug.py", line 19, in <module>
    topics, probs = topic_model.fit_transform(all_file_doc)
  File "/home/ai_user/.local/lib/python3.9/site-packages/bertopic/_bertopic.py", line 408, in fit_transform
    umap_embeddings = self._reduce_dimensionality(embeddings, y)
  File "/home/ai_user/.local/lib/python3.9/site-packages/bertopic/_bertopic.py", line 3355, in _reduce_dimensionality
    self.umap_model.fit(embeddings, y=y)
  File "/home/ai_user/.local/lib/python3.9/site-packages/cuml/internals/api_decorators.py", line 188, in wrapper
    ret = func(*args, **kwargs)
  File "/home/ai_user/.local/lib/python3.9/site-packages/cuml/internals/api_decorators.py", line 393, in dispatch
    return self.dispatch_func(func_name, gpu_func, *args, **kwargs)
  File "/home/ai_user/.local/lib/python3.9/site-packages/cuml/internals/api_decorators.py", line 190, in wrapper
    return func(*args, **kwargs)
  File "base.pyx", line 687, in cuml.internals.base.UniversalBase.dispatch_func
  File "umap.pyx", line 603, in cuml.manifold.umap.UMAP.fit
RuntimeError: cuBLAS error encountered at: file=/__w/cuml/cuml/python/_skbuild/linux-x86_64-3.9/cmake-build/_deps/raft-src/cpp/include/raft/core/resource/cublas_handle.hpp line=75: call='cublasSetStream(ret, get_cuda_stream(res))', Reason=1:CUBLAS_STATUS_NOT_INITIALIZED
Obtained 41 stack frames...

运行代码 #

import numpy as np
import pandas as pd
import os
import sys
import pickle as pkl
from bertopic import BERTopic
import cuml
os.environ['CUDA_VISIBLE_DEVICES']="1"

# GPU support
from cuml.cluster import HDBSCAN
from cuml.manifold import UMAP

# CPU (very slow if sample size is large)
#from umap import UMAP
#from hdbscan import HDBSCAN

all_file_doc = ['hi, good day!', 'how are you', 'abc def ghi', 'xyz 1234 qqq' ,'nasdaq amex', 'aapl msft']
umap_model = UMAP(n_components=5, n_neighbors=15, min_dist=0.0, random_state=42)
hdbscan_model = HDBSCAN(min_samples=10, gen_min_span_tree=True, prediction_data=True)
topic_model = BERTopic(umap_model=umap_model, hdbscan_model=hdbscan_model, nr_topics=3, calculate_probabilities=True)
topics, probs = topic_model.fit_transform(all_file_doc)

环境信息 #

• 系统：Ubuntu 20.04.3
• GPU：4块RTX3090
• NVIDIA驱动：470.223.02，CUDA版本11.4
• Python：3.9.5
• 关键冲突：BERTopic 0.16自动安装nvidia-cublas-cu12等CUDA12版本依赖，与本地CUDA11.4不兼容，降级BERTopic无效

核心是对齐CUDA版本，确保所有GPU依赖与本地CUDA11.4匹配，步骤如下：

1. 卸载冲突依赖
   先移除CUDA12版本相关包、cuml及BERTopic：

   pip uninstall -y bertopic cuml nvidia-cublas-cu12 nvidia-cudnn-cu12 nvidia-cuda-runtime-cu12
   

2. 安装适配CUDA11.4的cuml
   安装对应CUDA11.x版本的cuml，自动匹配兼容的CUDA依赖：

   pip install cuml-cu11 --extra-index-url=https://pypi.nvidia.com
   

3. 重新安装BERTopic并控制依赖
   禁止BERTopic自动安装依赖，手动补充必要组件：

   pip install bertopic --no-deps
   pip install numpy pandas scikit-learn sentence-transformers torch
   

4. 验证GPU环境
   在代码开头添加验证逻辑，确保cuBLAS正常加载：

   import torch
   # 检查CUDA可用性
   print(torch.cuda.is_available())
   # 验证cuBLAS初始化
   torch.backends.cuda.enable_cublas_fp16_reduced_precision_reduction(False)
   

5. 调整聚类参数（可选）
   示例样本量仅6条，HDBSCAN的min_samples=10参数不合理，建议调整为：

   hdbscan_model = HDBSCAN(min_samples=1, gen_min_span_tree=True, prediction_data=True)
   

内容的提问来源于stack exchange，提问作者jianfeng24