M1 Mac安装azureml-dataset-runtime失败,求解决pyarrow相关报错
问题描述
- 环境:M1芯片Mac,MacOS Sonoma 14.7,Python 3.8
- 故障场景:项目依赖在Windows可正常安装,但在本机安装时,
azureml-dataset-runtime依赖的pyarrow<4.0.0安装失败,核心原因是编译numpy==1.16.6时出现ModuleNotFoundError: No module named 'distutils.msvccompiler'(该模块为Windows专属)
项目依赖清单:
azureml azureml-core azureml-dataset-runtime azureml-interpret azureml-monitoring azureml-responsibleai azureml-telemetry azure-identity azure-mgmt-resource azureml-dataset-runtime numpy pandas>=1.5.0 pandera torch neuralforecast datasetsforecast hyperopt matplotlib darts mlflow
解决方案
方法1:通过pip强制使用预编译包
步骤1:安装适配M1的numpy预编译版本
numpy==1.16.6无M1预编译wheel,会触发源码编译并引入Windows依赖,先安装Python3.8兼容的M1适配版本:
pip install numpy==1.23.5 --only-binary :all:
步骤2:安装适配M1的pyarrow预编译版本
指定pyarrow<4.0.0范围内带M1预编译包的版本,避免源码编译:
pip install pyarrow==3.0.1 --only-binary :all:
步骤3:安装剩余依赖
完成上述两步后,安装全部项目依赖:
pip install azureml azureml-core azureml-dataset-runtime azureml-interpret azureml-monitoring azureml-responsibleai azureml-telemetry azure-identity azure-mgmt-resource pandas>=1.5.0 pandera torch neuralforecast datasetsforecast hyperopt matplotlib darts mlflow
方法2:使用conda管理(推荐,兼容性更好)
conda仓库提供完善的M1预编译包,避免源码编译问题:
- 创建并激活conda环境:
conda create -n ml_env python=3.8 conda activate ml_env
- 安装依赖:
conda install numpy pandas matplotlib pip install azureml azureml-core azureml-dataset-runtime azureml-interpret azureml-monitoring azureml-responsibleai azureml-telemetry azure-identity azure-mgmt-resource pandera torch neuralforecast datasetsforecast hyperopt darts mlflow
内容的提问来源于stack exchange,提问作者Furqan Hashim
相关产品推荐
相关产品推荐

