导入Dask DataFrame时遇AttributeError: pyarrow.lib无Bool8Type属性

在学生项目中导入Dask DataFrame时触发报错：AttributeError: module 'pyarrow.lib' has no attribute 'Bool8Type'。使用VS Code的conda环境Jupyter Notebook，已尝试更新、重装相关库但问题未解决。

代码 #

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import plotly.express as pxc
import re
from dask.distributed import LocalCluster, Client
from dask import dataframe as dd

报错输出 #

---------------------------------------------------------------------------

AttributeError                            Traceback (most recent call last)
Cell In[2], line 9
      7 import re
      8 from dask.distributed import LocalCluster, Client
----> 9 from dask import dataframe as dd

File c:\Users\tu\.conda\envs\pythonAD\Lib\site-packages\dask\dataframe\__init__.py:108
    105 except ImportError:
    106     pass
--> 108 if _dask_expr_enabled():
    109     import dask_expr as dd
    111     # trigger loading of dask-expr which will in-turn import dask.dataframe and run remainder
    112     # of this module's init updating attributes to be dask-expr
    113     # note: needs reload, in case dask-expr imported before dask.dataframe; works fine otherwise

File c:\Users\tu\.conda\envs\pythonAD\Lib\site-packages\dask\dataframe\__init__.py:33, in _dask_expr_enabled()
     31         return (DASK_EXPR_ENABLED := False)
     32     try:
--> 33         import dask_expr  # noqa: F401
     34     except ImportError:
     35         msg = """
     36 Dask dataframe query planning is disabled because dask-expr is not installed.
     37 
     38 You can install it with `pip install dask[dataframe]` or `conda install dask`.
     39 This will raise in a future version.
     40 """

File c:\Users\tu\.conda\envs\pythonAD\Lib\site-packages\dask_expr\__init__.py:3
      1 import dask.dataframe
--> 3 from dask_expr import _version, datasets
      4 from dask_expr._collection import *
      5 from dask_expr._dispatch import get_collection_type

File c:\Users\tu\.conda\envs\pythonAD\Lib\site-packages\dask_expr\datasets.py:8
      5 import pandas as pd
      6 from dask.dataframe.utils import pyarrow_strings_enabled
--> 8 from dask_expr._collection import new_collection
      9 from dask_expr._expr import ArrowStringConversion
     10 from dask_expr.io import BlockwiseIO, PartitionsFiltered

File c:\Users\tu\.conda\envs\pythonAD\Lib\site-packages\dask_expr\_collection.py:69
     67 from pandas.api.types import is_scalar as pd_is_scalar
     68 from pandas.api.types import is_timedelta64_dtype
--> 69 from pyarrow import fs as pa_fs
     70 from tlz import first
     72 import dask_expr._backends  # noqa: F401

File c:\Users\tu\.conda\envs\pythonAD\Lib\site-packages\pyarrow\fs.py:53
     50     _not_imported.append("HadoopFileSystem")
     52 try:
--> 53     from pyarrow._gcsfs import GcsFileSystem  # noqa
     54 except ImportError:
     55     _not_imported.append("GcsFileSystem")

File c:\Users\tu\.conda\envs\pythonAD\Lib\site-packages\pyarrow\_gcsfs.pyx:1, in init pyarrow._gcsfs()

AttributeError: module 'pyarrow.lib' has no attribute 'Bool8Type'

该报错源于pyarrow版本不兼容：Bool8Type是pyarrow较新版本新增的类型，当前环境中pyarrow版本过低或存在版本冲突。以下是具体解决步骤：

• 完全卸载重装pyarrow
  先清理残留的旧版本：

  pip uninstall -y pyarrow
  conda remove --force pyarrow
  

  再通过conda-forge安装适配版本：

  conda install pyarrow -c conda-forge
  

• 锁定dask与pyarrow的兼容版本
  若重装后问题仍存在，指定兼容版本组合安装：

  conda install dask[dataframe]=2024.5.0 pyarrow=16.1.0 -c conda-forge
  

  （版本号可根据官方兼容说明调整，确保两者版本匹配）

• 临时禁用dask-expr
  若无需dask-expr功能，可在导入dask前设置环境变量禁用：

  import os
  os.environ["DASK_EXPR_ENABLED"] = "False"
  
  # 后续正常导入dask模块
  from dask.distributed import LocalCluster, Client
  from dask import dataframe as dd
  

• 创建全新conda环境测试
  若环境存在损坏包，新建干净环境排查：

  conda create -n new_dask_env python=3.10
  conda activate new_dask_env
  conda install dask[dataframe] pyarrow seaborn matplotlib plotly -c conda-forge
  

内容的提问来源于stack exchange，提问作者Samuel Vranici