新conda环境中scikit-learn无法检测liblapack依赖的问题及解决问询

背景 #

在名为torch-nightly的conda环境中已安装scikit-learn，且环境内存在liblapack依赖：

(torch-nightly) au@a2 ~ % conda list |grep "scikit-learn"
scikit-learn              1.1.1            py38hc377ac9_0  
(torch-nightly) au@a2 ~ % conda list |grep "liblapack"
liblapack                 3.9.0           15_osxarm64_openblas    conda-forge

问题现象 #

执行import sklearn时触发导入错误，实际是scipy无法找到liblapack库：

(torch-nightly) au@a2 ~ % python -c "import sklearn"                                                               
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/sklearn/__init__.py", line 82, in <module>
    from .base import clone
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/sklearn/base.py", line 17, in <module>
    from .utils import _IS_32BIT
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/sklearn/utils/__init__.py", line 22, in <module>
    from scipy.sparse import issparse
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/__init__.py", line 283, in <module>
    from . import csgraph
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/csgraph/__init__.py", line 182, in <module>
    from ._laplacian import laplacian
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/csgraph/_laplacian.py", line 7, in <module>
    from scipy.sparse.linalg import LinearOperator
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/linalg/__init__.py", line 120, in <module>
    from ._isolve import *
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/linalg/_isolve/__init__.py", line 4, in <module>
    from .iterative import *
  File "/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/linalg/_isolve/iterative.py", line 9, in <module>
    from . import _iterative
ImportError: dlopen(/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/linalg/_isolve/_iterative.cpython-38-darwin.so, 0x0002): Library not loaded: '@rpath/liblapack.3.dylib'
  Referenced from: '/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/linalg/_isolve/_iterative.cpython-38-darwin.so'
  Reason: tried: '/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/linalg/_isolve/../../../../../../liblapack.3.dylib' (no such file), '/Users/au/miniconda3/envs/torch-nightly/lib/python3.8/site-packages/scipy/sparse/linalg/_isolve/../../../../../../liblapack.3.dylib' (no such file), '/Users/au/miniconda3/envs/torch-nightly/bin/../lib/liblapack.3.dylib' (no such file), '/Users/au/miniconda3/envs/torch-nightly/bin/../lib/liblapack.3.dylib' (no such file), '/usr/local/lib/liblapack.3.dylib' (no such file), '/usr/lib/liblapack.3.dylib' (no such file)

问题排查 #

确认报错中提到的liblapack.3.dylib实际存在于环境的lib目录：

(torch-nightly) au@a2 ~ % ls miniconda3/envs/torch-nightly/lib/liblapack.3.dylib
miniconda3/envs/torch-nightly/lib/liblapack.3.dylib

解决方法 #

方法1：强制重新安装兼容的scipy和scikit-learn包 #

问题根源大概率是不同conda channel的包混合导致的依赖路径不兼容，用conda-forge源重新安装相关包：

conda activate torch-nightly
conda install -c conda-forge scipy scikit-learn --force-reinstall

方法2：创建全新conda环境避免冲突 #

如果环境中存在其他冲突包，直接新建环境安装所需组件：

conda create -n new_torch_env python=3.8 torch-nightly scikit-learn -c pytorch -c conda-forge
conda activate new_torch_env

方法3：检查并修复库符号链接（可选） #

少数情况下可能是符号链接失效，检查链接指向：

ls -l /Users/au/miniconda3/envs/torch-nightly/lib/liblapack.3.dylib

如果链接指向错误，重新创建链接（替换实际库文件名）：

ln -sf /Users/au/miniconda3/envs/torch-nightly/lib/liblapack.3.dylib /Users/au/miniconda3/envs/torch-nightly/lib/liblapack.dylib

内容的提问来源于stack exchange，提问作者mkk