如何在Keras模型循环训练中正确更新DataFrame存储损失值?
问题原因分析
原代码的核心问题是错误地直接给DataFrame的列赋值单个值,而非为每组参数添加新的一行记录:
- 当
modeldata是空DataFrame时,执行modeldata['Epochs'] = e这类操作,无法生成有效行(单个值无法匹配空DataFrame的行长度)。 - 即使DataFrame有行,每次循环也会覆盖整列的所有值,最终只会保留最后一组循环的参数,而非所有组合的记录。
正确实现方式
推荐两种高效的实现方式:
方式1:先收集数据到列表,再生成DataFrame(性能更优)
这种方式避免频繁修改DataFrame,尤其适合参数组合较多的场景:
import pandas as pd epochs = [1,5,10,15,20,25,30] batch_sizes = [64,128,256,512] data_list = [] # 用列表存储每组参数的结果 for e in epochs: for bs in batch_sizes: # 训练模型 training = mod_nvp.fit( x_train, y_train, batch_size = bs, epochs = e, validation_split = 0.2, verbose = 0 ) y_pred = mod_nvp.predict(x_test, verbose = 0) current_loss = custom_loss_nvp1(y_test,y_pred) # 把当前参数和结果存入字典,再添加到列表 data_list.append({ 'Epochs': e, 'Batch Size': bs, 'Loss': current_loss, 'Training Loss': sum(training.history['loss']), 'Test Loss': sum(training.history['val_loss']) }) print('current running epoch',e,'with batchsize',bs) # 最后用列表生成完整DataFrame modeldata = pd.DataFrame(data_list)
方式2:逐行追加到DataFrame(适合小量数据)
如果参数组合较少,也可以每次循环生成单行DataFrame并追加:
import pandas as pd epochs = [1,5,10,15,20,25,30] batch_sizes = [64,128,256,512] modeldata = pd.DataFrame() for e in epochs: for bs in batch_sizes: training = mod_nvp.fit( x_train, y_train, batch_size = bs, epochs = e, validation_split = 0.2, verbose = 0 ) y_pred = mod_nvp.predict(x_test, verbose = 0) current_loss = custom_loss_nvp1(y_test,y_pred) # 生成单行DataFrame并追加 new_row = pd.DataFrame({ 'Epochs': [e], 'Batch Size': [bs], 'Loss': [current_loss], 'Training Loss': [sum(training.history['loss'])], 'Test Loss': [sum(training.history['val_loss'])] }) modeldata = pd.concat([modeldata, new_row], ignore_index=True) print('current running epoch',e,'with batchsize',bs)
额外注意事项
- 模型重置:如果
mod_nvp是在循环外定义的,每次训练会在上一次的基础上继续优化。若需要每组参数都训练全新模型,要把模型初始化代码(比如mod_nvp = build_your_model())放到循环内部。 - 损失值格式:确保
custom_loss_nvp1返回单个数值,而非数组,否则会出现DataFrame行长度不匹配的问题。
内容的提问来源于stack exchange,提问作者rex
相关产品推荐
相关产品推荐

