使用Pandas读取HDF5文件时遇ValueError错误的解决方法

尝试用Pandas读取HDF5文件时触发错误：ValueError: invalid literal for int() with base 10: '5,3,2'，仅调用data = pd.read_hdf(path)，不清楚该调整哪些参数。该文件可通过pyCGNS包正常读取，但希望用Pandas完成读取操作。完整报错堆栈如下：

runfile('D:/projekty/CGNS_Read/untitled3.py', wdir='D:/projekty/CGNS_Read')
Traceback (most recent call last):

  File C:\Program Files\Spyder\pkgs\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File d:\projekty\cgns_read\untitled3.py:19
    data = pd.read_hdf(path + name)

  File C:\Program Files\Spyder\pkgs\pandas\io\pytables.py:420 in read_hdf
    store = HDFStore(path_or_buf, mode=mode, errors=errors, **kwargs)

  File C:\Program Files\Spyder\pkgs\pandas\io\pytables.py:560 in __init__
    tables = import_optional_dependency("tables")

  File C:\Program Files\Spyder\pkgs\pandas\compat\_optional.py:142 in import_optional_dependency
    module = importlib.import_module(name)

  File importlib\__init__.py:127 in import_module

  File <frozen importlib._bootstrap>:1014 in _gcd_import

  File <frozen importlib._bootstrap>:991 in _find_and_load

  File <frozen importlib._bootstrap>:975 in _find_and_load_unlocked

  File <frozen importlib._bootstrap>:671 in _load_unlocked

  File <frozen importlib._bootstrap_external>:848 in exec_module

  File <frozen importlib._bootstrap>:219 in _call_with_frames_removed

  File C:\Program Files\Spyder\Python\lib\site-packages\tables\__init__.py:100
    from .file import File, open_file, copy_file

  File C:\Program Files\Spyder\Python\lib\site-packages\tables\file.py:20
    import numexpr as ne

  File C:\Program Files\Spyder\Python\lib\site-packages\numexpr\__init__.py:44
    nthreads = _init_num_threads()

  File C:\Program Files\Spyder\Python\lib\site-packages\numexpr\utils.py:156 in _init_num_threads
    requested_threads = int(os.environ['OMP_NUM_THREADS'])

ValueError: invalid literal for int() with base 10: '5,3,2'

• 问题根源：错误并非Pandas读取HDF5文件本身导致，而是numexpr库初始化时，尝试将环境变量OMP_NUM_THREADS的'5,3,2'值转为整数失败——该值是逗号分隔的字符串，无法直接转换为整数。
• 修复步骤：
  1. 在调用pd.read_hdf()前，修正OMP_NUM_THREADS环境变量为单个整数值，示例代码：

     import os
     # 根据自身CPU核心数设置合理线程数，比如4
     os.environ['OMP_NUM_THREADS'] = '4'
     

  2. 执行上述代码后再进行文件读取：

     data = pd.read_hdf(path + name)
     

• 额外说明：也可以直接检查系统或Spyder的环境变量配置，将OMP_NUM_THREADS的值改为合法的单个整数，避免逗号分隔格式，从根源解决问题。

内容的提问来源于stack exchange，提问作者Galedon