使用Conda环境基于Unsloth微调Llama3时遭遇bitsandbytes错误

问题描述 #

在Conda虚拟环境中使用Unsloth官方代码微调Llama3模型时，出现bitsandbytes无法加载CUDA库、退化为CPU版本，最终触发AttributeError: undefined symbol: cdequantize_blockwise_fp32错误，完整报错信息如下：

(unsloth_env) llm@llm:/mnt/ssd/unsloth$ python3 run_unsloth.py
WARNING: BNB_CUDA_VERSION=121 environment variable detected; loading libbitsandbytes_cuda121_nocublaslt121.so.
This can be used to load a bitsandbytes version that is different from the PyTorch CUDA version.
If this was unintended set the BNB_CUDA_VERSION variable to an empty string: export BNB_CUDA_VERSION=
If you use the manual override make sure the right libcudart.so is in your LD_LIBRARY_PATH
For example by adding the following to your .bashrc: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path_to_cuda_dir/lib64

Could not find the bitsandbytes CUDA binary at PosixPath('/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121_nocublaslt121.so')
The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/__init__.py:72: UserWarning: Unsloth: Running `ldconfig /usr/lib64-nvidia` to link CUDA.
  warnings.warn(
/sbin/ldconfig.real: Can't create temporary cache file /etc/ld.so.cache~: Permission denied
/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/__init__.py:103: UserWarning: Unsloth: CUDA is not linked properly.
Try running `python -m bitsandbytes` then `python -m xformers.info`
We tried running `ldconfig /usr/lib64-nvidia` ourselves, but it didn't work.
You need to run in your terminal `sudo ldconfig /usr/lib64-nvidia` yourself, then import Unsloth.
Also try `sudo ldconfig /usr/local/cuda-xx.x` - find the latest cuda version.
Unsloth will still run for now, but maybe it might crash - let's hope it works!
  warnings.warn(
Traceback (most recent call last):
  File "/mnt/ssd/unsloth/run_unsloth.py", line 2, in <module>
    import unsloth
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/__init__.py", line 113, in <module>
    from .models import *
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/models/__init__.py", line 15, in <module>
    from .loader import FastLanguageModel
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/models/loader.py", line 15, in <module>
    from .llama import FastLlamaModel, logger
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/models/llama.py", line 26, in <module>
    from ..kernels import *
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/kernels/__init__.py", line 15, in <module>
    from .cross_entropy_loss import fast_cross_entropy_loss
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/kernels/cross_entropy_loss.py", line 18, in <module>
    from .utils import calculate_settings, MAX_FUSED_SIZE
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/unsloth/kernels/utils.py", line 36, in <module>
    cdequantize_blockwise_fp32      = bnb.functional.lib.cdequantize_blockwise_fp32
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 73, in __getattr__
    return getattr(self._lib, item)
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/home/llm/miniconda3/envs/unsloth_env/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /home/llm/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cdequantize_blockwise_fp32

解决方案 #

1. 修复CUDA库链接权限问题 #

执行以下命令手动链接CUDA库（替换cuda-12.1为你的实际CUDA版本）：

sudo ldconfig /usr/lib64-nvidia
sudo ldconfig /usr/local/cuda-12.1/lib64

这一步解决Unsloth自动执行ldconfig时的权限拒绝问题，确保系统能找到CUDA相关库。

2. 重新安装匹配CUDA版本的bitsandbytes #

• 先清除强制指定的BNB_CUDA_VERSION环境变量：

  export BNB_CUDA_VERSION=
  

• 卸载当前不兼容的bitsandbytes：

  pip uninstall -y bitsandbytes
  

• 安装与你的CUDA版本匹配的bitsandbytes预编译包（以CUDA 12.1为例，选择对应版本）：

  pip install bitsandbytes==0.42.0 --force-reinstall
  

• 如果预编译包仍不兼容，从源码编译bitsandbytes：

  git clone https://github.com/TimDettmers/bitsandbytes.git
  cd bitsandbytes
  export CUDA_HOME=/usr/local/cuda-12.1
  python setup.py install
  

3. 验证环境配置 #

完成上述步骤后，验证各组件的CUDA支持：

• 检查bitsandbytes的CUDA检测：

  python -m bitsandbytes
  

• 检查xformers的CUDA支持：

  python -m xformers.info
  

• 确认PyTorch可用CUDA：

  python -c "import torch; print(torch.cuda.is_available())"
  

内容的提问来源于stack exchange，提问作者cerebro007