MacBook Air(2020)导入tensorflow_datasets触发ImportError求助

环境信息 #

• 设备：2020款MacBook Air（M1芯片）
• Python版本：3.10.9
• 相关依赖版本：
  • tensorflow-macos：2.9.1
  • tensorflow-metal：0.5.1
  • tensorflow-datasets：4.9.0
• 运行环境：Miniconda创建的虚拟环境

报错详情 #

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[2], line 3
      1 import tensorflow as tf
      2 # Brining in tensorflow datasets for fashion mnist 
----> 3 import tensorflow_datasets as tfds
      4 # Bringing in matplotlib for viz stuff
      5 from matplotlib import pyplot as plt

File ~/miniconda/envs/env-tf2/lib/python3.10/site-packages/tensorflow_datasets/__init__.py:43
     41 _TIMESTAMP_IMPORT_STARTS = time.time()
     42 from absl import logging
----> 43 import tensorflow_datasets.core.logging as _tfds_logging
     44 from tensorflow_datasets.core.logging import call_metadata as _call_metadata
     46 _metadata = _call_metadata.CallMetadata()

File ~/miniconda/envs/env-tf2/lib/python3.10/site-packages/tensorflow_datasets/core/__init__.py:22
     18 # Allow to use `tfds.core.Path` in dataset implementation which seems more
     19 # natural than having to import a third party module.
     20 from etils.epath import Path
----> 22 from tensorflow_datasets.core import community
     23 from tensorflow_datasets.core.dataset_builder import BeamBasedBuilder
     24 from tensorflow_datasets.core.dataset_builder import BuilderConfig

File ~/miniconda/envs/env-tf2/lib/python3.10/site-packages/tensorflow_datasets/core/community/__init__.py:18
      1 # coding=utf-8
      2 # Copyright 2023 The TensorFlow Datasets Authors.
      3 #
   (...)
     13 # See the License for the specific language governing permissions and
     14 # limitations under the License.
     16 \"\"\"Community dataset API.\"\"\"
----> 18 from tensorflow_datasets.core.community.huggingface_wrapper import mock_builtin_to_use_gfile
     19 from tensorflow_datasets.core.community.huggingface_wrapper import mock_huggingface_import
     20 from tensorflow_datasets.core.community.load import builder_cls_from_module

File ~/miniconda/envs/env-tf2/lib/python3.10/site-packages/tensorflow_datasets/core/community/huggingface_wrapper.py:31
     28 from unittest import mock
     30 from etils import epath
----> 31 from tensorflow_datasets.core import dataset_builder
     32 from tensorflow_datasets.core import dataset_info
     33 from tensorflow_datasets.core import download

File ~/miniconda/envs/env-tf2/lib/python3.10/site-packages/tensorflow_datasets/core/dataset_builder.py:34
     32 from etils import epath
     33 from tensorflow_datasets.core import constants
----> 34 from tensorflow_datasets.core import dataset_info
     35 from tensorflow_datasets.core import dataset_metadata
     36 from tensorflow_datasets.core import decode

File ~/miniconda/envs/env-tf2/lib/python3.10/site-packages/tensorflow_datasets/core/dataset_info.py:47
     45 from etils import epath
     46 from tensorflow_datasets.core import constants
----> 47 from tensorflow_datasets.core import file_adapters
     48 from tensorflow_datasets.core import lazy_imports_lib
     49 from tensorflow_datasets.core import naming

File ~/miniconda/envs/env-tf2/lib/python3.10/site-packages/tensorflow_datasets/core/file_adapters.py:29
     26 from tensorflow_datasets.core.utils import type_utils
     27 from tensorflow_datasets.core.utils.lazy_imports_utils import tensorflow as tf
----> 29 from array_record.python import array_record_module
     31 ExamplePositions = List[Any]
     34 class FileFormat(enum.Enum):

ImportError: dlopen(/Users/lars/miniconda/envs/env-tf2/lib/python3.10/site-packages/array_record/python/array_record_module.so, 0x0002): tried: '/Users/lars/miniconda/envs/env-tf2/lib/python3.10/site-packages/array_record/python/array_record_module.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/lars/miniconda/envs/env-tf2/lib/python3.10/site-packages/array_record/python/array_record_module.so' (no such file), '/Users/lars/miniconda/envs/env-tf2/lib/python3.10/site-packages/array_record/python/array_record_module.so' (not a mach-o file)

解决方案 #

这个错误是因为array_record包安装了x86架构版本，与M1芯片的arm64架构不兼容导致的，按以下步骤修复：

1. 确保conda环境为arm64架构

   • 未创建环境的话，直接指定架构创建：

     CONDA_SUBDIR=osx-arm64 conda create -n env-tf2 python=3.10
     

   • 已有环境的话，激活后修改架构配置：

     conda activate env-tf2
     conda config --env --set subdir osx-arm64
     

2. 重新安装tensorflow-datasets及依赖
   先卸载现有包：

   pip uninstall -y tensorflow-datasets array_record
   

   再安装适配arm64的版本：

   pip install tensorflow-datasets==4.9.0
   

3. 验证修复结果
   激活环境后运行以下代码：

   import tensorflow as tf
   import tensorflow_datasets as tfds
   

   无报错即表示修复成功。

额外提示：tensorflow-macos 2.9.1与Python 3.10的兼容性有限，也可以考虑升级tensorflow-macos到2.10及以上版本，同时对应升级tensorflow-metal（需保持版本匹配）。

内容的提问来源于stack exchange，提问作者Lars B.