使用NCO转换NetCDF3至NetCDF4时遭遇HDF5层错误求助

执行了以下NCO命令处理NetCDF文件：

ncks -C -v expver in.nc expver.nc

ncks -5 -C -x -v expver in.nc in_5.nc

ncrename -d latitude,lat -d longitude,lon -d valid_time,time
         -v latitude,lat -v longitude,lon -v valid_time,time
         in_5.nc

ncks -4 in_5.nc in_4.nc

ncks -A -v expver expver.nc in_4.nc

修正ncrename的拼写错误后执行，出现如下HDF5层错误：

ERROR NC_EHDFERR Error at HDF5 layer
HINT: NC_EHDFERR errors indicate that the HDF5-backend to netCDF is unable to perform the requested task. NCO can receive this devilishly inscrutable error for a variety of possible reasons including: 1) The run-time dynamic linker attempts to resolve calls from the netCDF library to the HDF library with an HDF5 libhdf5.a that is incompatible with the version used to build NCO and netCDF. 2) The file system does not allow the HDF5 flock() function, as of HDF5 1.10.x, to enable multiple processes to open the same file for reading, a feature known as SWMR (Single Write Multiple Read). The fix is to disable the HDF5 flock() by setting an environment variable thusly: "export HDF5_USE_FILE_LOCKING=FALSE". 3) An incorrect netCDF4 library implementation of a procedure (e.g., nc_rename_var()) in terms of HDF function calls (e.g., HDF5Lmove()) manifests an error or inconsistent state within the HDF5 layer. This often occurs during renaming operations (https://github.com/Unidata/netcdf-c/issues/597). 4) Attempting to compress or decompress a netCDF4 dataset with a non-standard (i.e., non-DEFLATE) filter when the requisite shared library to encode/decode that compression filter is not present in either the default location (/usr/local/hdf5/lib/plugin) or in the user-configurable location referred to by the HDF5_PLUGIN_PATH environment variable. One can determine if missing plugin libraries are the culprit by dumping the hidden attributes of the dataset with, e.g., ncks --hdn -m in.nc or ncdump -s -h in.nc. Any variables with (hidden) "_Filter" attributes require the corresponding shared libraries to be located in HDF5_PLUGIN_PATH. Some HDF5 implementations (at least MacOSX with MacPorts as of 20200907) may also require explicitly setting the plugin path in the environment, even for the default location! To test this, re-try your NCO command after doing this: "export HDF5_PLUGIN_PATH=/usr/local/hdf5/lib/plugin". 5) Bad vibes.
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_enddef()
nco_err_exit(): ERROR Error code is -101. Translation into English with nc_strerror(-101) is "NetCDF: HDF error"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

已确认：无隐藏变量、重命名操作本身成功、文件未被锁定，寻求解决办法。

结合错误提示和你的排查结果，可尝试以下几种针对性解决办法：

• 强制关闭HDF5文件锁：部分文件系统可能存在锁机制的隐性问题，即使你认为文件未被锁定。执行以下命令后重新运行ncrename：

  export HDF5_USE_FILE_LOCKING=FALSE
  

  若要永久生效，可将该命令添加到shell配置文件（如~/.bashrc或~/.zshrc）。

• 规避NetCDF4重命名底层bug：重命名操作可能触发NetCDF4的HDF5调用bug，可通过格式转换绕开：

  # 转成NetCDF3格式
  ncks -3 in_5.nc in_3.nc
  # 执行重命名
  ncrename -d latitude,lat -d longitude,lon -d valid_time,time -v latitude,lat -v longitude,lon -v valid_time,time in_3.nc
  # 转回NetCDF4格式
  ncks -4 in_3.nc in_5_fixed.nc
  

  之后用in_5_fixed.nc继续后续的ncks操作。

• 检查库版本兼容性：验证NetCDF、HDF5和NCO的版本是否匹配，执行以下命令查看版本信息：

  ncks --version
  nc-config --version
  h5dump --version
  

  若存在版本不兼容，需重新安装匹配版本的依赖库或NCO。

• 重新生成中间文件：旧的in_5.nc可能因之前错误操作留下隐性损坏，重新生成后再执行重命名：

  rm in_5.nc
  ncks -5 -C -x -v expver in.nc in_5.nc
  ncrename -d latitude,lat -d longitude,lon -d valid_time,time -v latitude,lat -v longitude,lon -v valid_time,time in_5.nc
  

内容的提问来源于stack exchange，提问作者ᚠᚱᛇᚺᛖᚱᚱᛖ