使用xarray.open_mfdataset合并NetCDF文件时遇dask块管理器识别错误

问题代码 #

nc_all = xarray.open_mfdataset(files,combine = 'nested', concat_dim="time")
files = glob.glob("/filepath/*")

报错信息 #

Traceback (most recent call last):
  File "/home/lsrathore/GLEAM/GLEAM_HPC.py", line 85, in <module>
    nc_1980_90 = xarray.open_mfdataset(files[1:11],combine = 'nested', concat_dim="time")
  File "/home/lsrathore/.local/lib/python3.9/site-packages/xarray/backends/api.py", line 1038, in open_mfdataset
    datasets = [open_(p, **open_kwargs) for p in paths]
  File "/home/lsrathore/.local/lib/python3.9/site-packages/xarray/backends/api.py", line 1038, in <listcomp>
    datasets = [open_(p, **open_kwargs) for p in paths]
  File "/home/lsrathore/.local/lib/python3.9/site-packages/xarray/backends/api.py", line 572, in open_dataset
    ds = _dataset_from_backend_dataset(
  File "/home/lsrathore/.local/lib/python3.9/site-packages/xarray/backends/api.py", line 367, in _dataset_from_backend_dataset
    ds = _chunk_ds(
  File "/home/lsrathore/.local/lib/python3.9/site-packages/xarray/backends/api.py", line 315, in _chunk_ds
    chunkmanager = guess_chunkmanager(chunked_array_type)
  File "/home/lsrathore/.local/lib/python3.9/site-packages/xarray/core/parallelcompat.py", line 87, in guess_chunkmanager
    raise ValueError(
ValueError: unrecognized chunk manager dask - must be one of: []

错误成因 #

• Dask依赖缺失或损坏：Xarray处理多NetCDF文件合并时，默认依赖Dask管理分块数组。当前Python环境中未安装Dask，或者Dask安装不完整、文件损坏，导致Xarray无法识别Dask作为分块管理器，触发该错误。
• 代码执行顺序错误：你提供的代码中，先调用xarray.open_mfdataset传入files变量，之后才通过glob.glob定义files，这会导致第一次调用时files未定义。虽然报错来自files[1:11]的调用，但如果实际代码存在变量未提前定义的情况，也会引发逻辑问题。
• 版本不兼容：旧版本Xarray对Dask的依赖处理存在缺陷，或者环境中Xarray与Dask版本不匹配，导致Xarray无法正确加载Dask的分块管理器。

内容的提问来源于stack exchange，提问作者lsr729