WSL Ubuntu下h5py驱动锁请求失败问题及解决咨询
# write data if use_hdf5: exp_pw_df.to_hdf(case_file, f"sample_{isample}/exp_pw") ... f = h5py.File(case_file, 'a') if 'exp_cov' in f[f"sample_{isample}"].keys(): del f[f'sample_{isample}/exp_cov'] else: pass f.create_dataset(f'sample_{isample}/exp_cov', data=CovT) f.close() else: exp_pw_df.to_csv(os.path.join(case_file,f'sample_{isample}','exp_pw'), index=False) theo_pw_df.to_csv(os.path.join(case_file,f'sample_{isample}','theo_pw'), index=False)
--- # 问题原因与修复方案 ## 核心原因 代码存在**重复打开文件句柄**的问题:主逻辑中已经通过`h5py.File(case_file, "a")`打开了文件,但在`sample_and_write`函数里,又调用`exp_pw_df.to_hdf()`和`h5py.File(case_file, 'a')`再次打开同一个文件。HDF5默认的文件锁机制不允许同一进程多次以可写模式打开同一文件,直接触发锁请求失败,抛出`errno=11`资源不可用错误。 另外,WSL2的文件系统(尤其是挂载Windows磁盘时)对文件锁的支持存在兼容性缺陷,会放大这类锁冲突的出现概率。 ## 修复方法 ### 方法1:传递已打开的文件句柄(最优解) 重构代码,避免重复打开文件,直接将主逻辑中已打开的`h5f`句柄传给`sample_and_write`函数,全程复用同一个文件句柄: 1. 修改主逻辑调用处: ```python if use_hdf5: check_case_file(case_file) # 使用with语句自动管理文件关闭,避免漏关句柄 with h5py.File(case_file, "a") as h5f: for i in range(min(dataset_range), max(dataset_range)): sample_group = f'sample_{i}' if sample_group in h5f: if ('exp_pw' in h5f[sample_group]) and ('theo_pw' in h5f[sample_group]) and ('theo_par' in h5f[sample_group]): if overwrite: sample_and_write(h5f, i, particle_pair, experiment, solver, open_data, fixed_resonance_ladder, vary_Erange, use_hdf5) else: samples_not_being_generated.append(i) else: sample_and_write(h5f, i, particle_pair, experiment, solver, open_data, fixed_resonance_ladder, vary_Erange, use_hdf5) else: sample_and_write(h5f, i, particle_pair, experiment, solver, open_data, fixed_resonance_ladder, vary_Erange, use_hdf5)
- 修改
sample_and_write函数:
def sample_and_write(h5f, isample, particle_pair, experiment, solver, open_data, fixed_resonance_ladder, vary_Erange, use_hdf5): ... # write data if use_hdf5: # 基于已打开的文件句柄,用pandas的HDFStore写入数据 with pd.HDFStore(h5f.filename, mode='a', driver='H5FD_CORE', fs=h5f.id) as store: store.put(f"sample_{isample}/exp_pw", exp_pw_df) # 直接通过已有句柄操作数据集,无需重新打开文件 sample_group = h5f.require_group(f'sample_{isample}') if 'exp_cov' in sample_group: del sample_group['exp_cov'] sample_group.create_dataset('exp_cov', data=CovT) else: exp_pw_df.to_csv(os.path.join(case_file,f'sample_{isample}','exp_pw'), index=False) theo_pw_df.to_csv(os.path.join(case_file,f'sample_{isample}','theo_pw'), index=False)
方法2:临时关闭句柄再调用函数
如果不想大幅重构代码,可以在调用sample_and_write前关闭主逻辑的文件句柄,调用完成后重新打开:
if use_hdf5: check_case_file(case_file) for i in range(min(dataset_range), max(dataset_range)): h5f = h5py.File(case_file, "a") sample_group = f'sample_{i}' need_write = False if sample_group in h5f: if ('exp_pw' in h5f[sample_group]) and ('theo_pw' in h5f[sample_group]) and ('theo_par' in h5f[sample_group]): if overwrite: need_write = True else: samples_not_being_generated.append(i) else: need_write = True else: need_write = True h5f.close() # 先关闭文件,避免锁冲突 if need_write: sample_and_write(case_file, i, particle_pair, experiment, solver, open_data, fixed_resonance_ladder, vary_Erange, use_hdf5)
方法3:禁用文件锁(仅临时调试用)
打开文件时添加lock=False参数跳过锁机制,但可能导致多进程/线程写入时的数据损坏,不推荐生产环境使用:
h5f = h5py.File(case_file, "a", lock=False)
是否需要补充其他信息?
目前提供的错误日志、环境配置和代码片段已经足够定位问题。如果修复后仍有异常,可以补充:
check_case_file函数的具体实现- pandas和h5py的版本号
- HDF5文件是否存储在WSL2挂载的Windows磁盘分区上
内容的提问来源于stack exchange,提问作者twistfire
相关产品推荐
相关产品推荐

