M1芯片Mac运行AgglomerativeClustering时Python内核频繁崩溃求助

问题背景 #

使用Python 3.10 + Anaconda包管理器，在Spyder和VSCode中运行sklearn.AgglomerativeClustering聚类代码时，单元格执行35-45秒后内核意外崩溃重启。Spyder未提供崩溃细节，VSCode仅提示内核崩溃相关日志。已尝试新建conda虚拟环境重装numpy、pandas、scikit-learn等依赖库，问题未解决。设备为M1芯片Mac，怀疑是芯片兼容性问题，但未正确应用scikit-learn官方文档中M1相关适配内容。

崩溃触发代码 #

aggclus = AgglomerativeClustering(n_clusters = 4, affinity="euclidean",linkage="ward") 
subset_pcafeatures = pca_features[:5000,5]  # 仅定义未实际使用
cluster_labels = aggclus.fit_predict(pca_features)

注：pca_features是原始数据经PCA.fit_transform生成的numpy数组，PCA执行无异常；尝试使用前5000行数据子集测试，仍出现内核崩溃。

错误提示信息 #

The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. View Jupyter log for further details.
Canceled future for execute_request message before replies were done

环境配置（2023年1月29日更新） #

执行conda list输出：

# packages in environment at ../anaconda3/envs/work-env:
#
# Name                    Version                   Build  Channel
appnope                   0.1.3              pyhd8ed1ab_0    conda-forge
asttokens                 2.2.1              pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
blas                      2.116                  openblas    conda-forge
blas-devel                3.9.0           16_osxarm64_openblas    conda-forge
bottleneck                1.3.5           py310h96f19d2_0  
brotli                    1.0.9                h1a28f6b_7  
brotli-bin                1.0.9                h1a28f6b_7  
bzip2                     1.0.8                h620ffc9_4  
ca-certificates           2023.01.10           hca03da5_0  
certifi                   2022.12.7       py310hca03da5_0  
comm                      0.1.2              pyhd8ed1ab_0    conda-forge
contourpy                 1.0.5           py310h525c30c_0  
cycler                    0.11.0             pyhd3eb1b0_0  
debugpy                   1.5.1           py310hc377ac9_0  
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
fftw                      3.3.9                h1a28f6b_1  
fonttools                 4.25.0             pyhd3eb1b0_0  
freetype                  2.12.1               h1192e45_0  
giflib                    5.2.1                h80987f9_1  
ipykernel                 6.20.2             pyh736e0ef_0    conda-forge
ipython                   8.8.0              pyhd1c38e8_0    conda-forge
jedi                      0.18.2             pyhd8ed1ab_0    conda-forge
joblib                    1.1.1           py310hca03da5_0  
jpeg                      9e                   h1a28f6b_0  
jupyter_client            7.4.9              pyhd8ed1ab_0    conda-forge
jupyter_core              5.1.1           py310hca03da5_0  
kiwisolver                1.4.4           py310h313beb8_0  
lcms2                     2.12                 hba8e193_0  
lerc                      3.0                  hc377ac9_0  
libblas                   3.9.0           16_osxarm64_openblas    conda-forge
libbrotlicommon           1.0.9                h1a28f6b_7  
libbrotlidec              1.0.9                h1a28f6b_7  
libbrotlienc              1.0.9                h1a28f6b_7  
libcblas                  3.9.0           16_osxarm64_openblas    conda-forge
libcxx                    14.0.6               h848a8c0_0  
libdeflate                1.8                  h1a28f6b_5  
libffi                    3.4.2                hca03da5_6  
libgfortran               5.0.0           11_3_0_hca03da5_28  
libgfortran5              11.3.0              h009349e_28  
liblapack                 3.9.0           16_osxarm64_openblas    conda-forge
liblapacke                3.9.0           16_osxarm64_openblas    conda-forge
libopenblas               0.3.21          openmp_hc731615_3    conda-forge
libpng                    1.6.37               hb8d0fd4_0  
libsodium                 1.0.18               h27ca646_1    conda-forge
libtiff                   4.5.0                h2fd578a_0  
libwebp                   1.2.4                h68602c7_0  
libwebp-base              1.2.4                h1a28f6b_0  
llvm-openmp               14.0.6               hc6e5704_0  
lz4-c                     1.9.4                h313beb8_0  
matplotlib                3.6.2           py310hca03da5_0  
matplotlib-base           3.6.2           py310h8bbb115_0  
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
missingno                 0.4.2              pyhd3eb1b0_1  
munkres                   1.1.4                      py_0  
ncurses                   6.4                  h313beb8_0  
nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
numexpr                   2.8.4           py310hecc3335_0  
numpy                     1.23.5          py310hb93e574_0  
numpy-base                1.23.5          py310haf87e8b_0  
openblas                  0.3.21          openmp_hf78f355_3    conda-forge
openssl                   1.1.1s               h1a28f6b_0  
packaging                 23.0               pyhd8ed1ab_0    conda-forge
pandas                    1.5.2           py310h46d7db6_0  
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.3.0           py310hf4a492f_1  
pip                       22.3.1          py310hca03da5_0  
platformdirs              2.6.2              pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.36             pyha770c72_0    conda-forge
psutil                    5.9.0           py310h1a28f6b_0  
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.14.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9           py310hca03da5_0  
python                    3.10.9               hc0d8a6c_0  
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
pytz                      2022.7          py310hca03da5_0  
pyzmq                     23.2.0          py310hc377ac9_0  
readline                  8.2                  h1a28f6b_0  
scikit-learn              1.2.0           py310h313beb8_0  
scipy                     1.9.3           py310h20cbe94_0  
seaborn                   0.12.2          py310hca03da5_0  
setuptools                65.6.3          py310hca03da5_0  
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.40.1               h7a7dc30_0  
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
threadpoolctl             2.2.0              pyh0d69192_0  
tk                        8.6.12               hb8d0fd4_0  
tornado                   6.2             py310h1a28f6b_0  
traitlets                 5.8.1              pyhd8ed1ab_0    conda-forge
typing-extensions         4.4.0                hd8ed1ab_0    conda-forge
typing_extensions         4.4.0              pyha770c72_0    conda-forge
tzdata                    2022g                h04d1e81_0  
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
wheel                     0.37.1             pyhd3eb1b0_0  
xz                        5.2.10               h80987f9_1  
zeromq                    4.3.4                hbdafb3b_1    conda-forge
zlib                      1.2.13               h5a0b063_0  
zstd                      1.5.2                h8574219_0  

解决建议 #

1. 切换BLAS后端至Apple Accelerate #

M1芯片对OpenBLAS兼容性存在问题，尝试使用Apple原生Accelerate框架：

• 新建conda环境并指定依赖：

  conda create -n sklearn-env python=3.10
  conda activate sklearn-env
  conda install -c conda-forge scikit-learn numpy scipy pandas "blas=*=accelerate"
  

• 验证后端：运行代码确认是否使用Apple Accelerate

  import numpy as np
  print(np.__config__.show())
  import sklearn
  print(sklearn.show_versions())
  

2. 调整数据规模与聚类参数 #

• 测试更小数据子集（如前1000行），排除内存过载问题
• 更换linkage参数为single或complete，ward linkage计算复杂度更高，易触发兼容性问题

3. 通过Rosetta 2运行环境 #

• 右键终端→显示简介→勾选"使用Rosetta打开"
• 在该终端中创建新环境并安装依赖，以x86_64兼容模式运行代码

4. 验证其他聚类算法 #

先测试KMeans是否正常运行，确认是算法问题还是环境问题：

from sklearn.cluster import KMeans
kmeans = KMeans(n_clusters=4, random_state=42)
cluster_labels = kmeans.fit_predict(pca_features)

5. 更新scikit-learn版本 #

当前使用1.2.0版本，尝试更新至最新稳定版：

conda update -c conda-forge scikit-learn

内容的提问来源于stack exchange，提问作者13R1_