解决Keras交叉验证保存模型时的ValueError：数据集名称已存在

问题概况 #

使用StratifiedShuffleSplit进行10折交叉验证时，第二折训练完成后调用model.save()总会触发报错，错误信息为ValueError: Unable to create dataset (name already exists)。已尝试删除旧模型文件、升级/降级依赖库（当前环境：h5py 3.9.0，keras 2.8.0，tensorflow 2.8.0），但问题仍未解决。单折训练耗时12小时，错误导致大量时间浪费。

相关代码片段 #

gc.collect()
sss = StratifiedShuffleSplit(n_splits=10, test_size=0.3, random_state=0)
fold_no = 1
annealer = LearningRateScheduler(lambda x: 1e-3 * 0.9 ** x)
callback2 = CustomEarlyStopping(patience=7)#100)                             
optimizer = keras.optimizers.Adam(learning_rate=1e-4)
acc_per_fold,loss_per_fold = [],[]
needTrain=True
for train_index, test_index in sss.split(X, y):
    # if fold_no > 1:
    clear_session()
    gc.collect()
    model = build_model(
      X.shape,             
      numClass,
      ) 
    model.compile(loss = 'categorical_crossentropy',
          optimizer=optimizer,
          metrics=['accuracy'])        
    nmModel = 'model_overlap_%d_%d_fold%d.h5'%(n_time_steps,step,fold_no)
    print('------------------------------------------------------------------------')
    print(f'Training for fold {fold_no} ...')
    training_generator = BalancedDataGenerator(X[train_index],
                                        y[train_index],                                                   
                                        batch_size=256)                   

    if needTrain:
        history =  model.fit( 
            training_generator,       
            epochs=1000,callbacks=[
                           callback2,
                            annealer
                           ], verbose=1,                           
            validation_data = (X[test_index],y[test_index]),
            )  
        # model.save(nmModel)     
        if os.path.exists(nmModel):
            os.remove(nmModel)
        model.save(nmModel)
   
    model.load_weights(nmModel)
    scores = model.evaluate(X[test_index],y[test_index], verbose=0)
    print(f'Score for fold {fold_no}: {model.metrics_names[0]} of {scores[0]}; {model.metrics_names[1]} of {scores[1]*100}%')
    acc_per_fold.append(scores[1] * 100)
    loss_per_fold.append(scores[0])    
    # Increase fold number
    fold_no = fold_no + 1
    del model
    gc.collect()

报错栈信息 #

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

File d:\tuh3salman\trainmodeloverlapseqbuku_all.py:298
    model.save(nmModel)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\keras\utils\traceback_utils.py:67 in error_handler
    raise e.with_traceback(filtered_tb) from None

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\h5py\_hl\group.py:183 in create_dataset
    dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\h5py\_hl\dataset.py:163 in make_new_dset
    dset_id = h5d.create(parent.id, name, tid, sid, dcpl=dcpl, dapl=dapl)

File h5py\_objects.pyx:54 in h5py._objects.with_phil.wrapper

File h5py\_objects.pyx:55 in h5py._objects.with_phil.wrapper

File h5py\h5d.pyx:138 in h5py.h5d.create

ValueError: Unable to create dataset (name already exists)

解决思路 #

1. 优化模型保存的文件操作逻辑 #

原代码中先删除旧文件再保存的方式可能存在竞态条件，或者HDF5文件句柄未完全释放。建议改用临时文件过渡的方式：

import shutil

# 替换原保存代码段
temp_path = nmModel + ".temp"
model.save(temp_path)
if os.path.exists(nmModel):
    os.remove(nmModel)
shutil.move(temp_path, nmModel)

2. 改用SavedModel格式替代HDF5 #

TensorFlow的SavedModel格式比HDF5更稳定，可避免部分HDF5的文件冲突问题：

# 替换文件名和保存方式
model_dir = f"model_overlap_{n_time_steps}_{step}_fold{fold_no}"
model.save(model_dir)
# 加载时用
model = tf.keras.models.load_model(model_dir)

3. 检查build_model中的层命名 #

如果build_model函数中手动给层指定了固定的name参数，多次创建模型时会导致重复命名，进而在保存HDF5时冲突。确保层使用自动命名，或者动态生成唯一名称：

# 示例：动态生成层名
dense_layer = Dense(64, name=f'dense_layer_{fold_no}')

4. 显式清理HDF5文件句柄 #

在删除旧模型文件前，显式确保所有HDF5相关句柄关闭：

import h5py

# 替换原删除文件的代码
if os.path.exists(nmModel):
    try:
        with h5py.File(nmModel, 'r') as f:
            pass  # 强制打开后关闭，释放句柄
        os.remove(nmModel)
    except:
        pass
model.save(nmModel)

内容的提问来源于stack exchange，提问作者Salman Alfariq