使用scikeras.wrappers.KerasRegressor调参时遇指标识别错误

尝试使用scikeras.wrappers.KerasRegressor进行超参数调优时，出现指标解析错误，代码及错误信息如下：

代码 #

# define a func to create an instance of lstm_model
def create_lstm_model():
    
    model = Sequential([
            LSTM(5, input_shape = (Xtrain.shape[1], Xtrain.shape[2]), dropout = 0.1, activation = 'tanh', return_sequences = True),
            LSTM(10, dropout = 0.05, activation = 'tanh'),
            Dense(5, activation = 'relu'),
            Dense(1)
        ])
    model.compile(optimizer = tf.keras.optimizers.Adam(), loss = tf.keras.losses.MeanSquaredError(), metrics = [keras.metrics.MeanSquaredError()])
    
    return model

#create the sklearn model for the network
model = KerasRegressor(build_fn = create_lstm_model, verbose = 1)

#param grid
batches = [16, 32]
epochs = [3, 4]

param_grid = dict(batch_size = batches, epochs = epochs)

grid = GridSearchCV(estimator = model,
                    param_grid = param_grid,
                    cv = 3)
grid.fit(Xtrain, ytrain, validation_data = (Xvalidation, yvalidation))

错误信息 #

fn_or_cls = keras_metric_get(metric)
  File "/home/aaa/Desktop/aaa/aaa/2024-gold-price-prediction-with-lstm-model/.venv/lib/python3.10/site-packages/keras/src/metrics/__init__.py", line 204, in get
    raise ValueError(f"Could not interpret metric identifier: {identifier}")
ValueError: Could not interpret metric identifier: loss

已尝试简化代码、移除metrics参数，但问题仍未解决。

核心原因 #

GridSearchCV本身会自动执行交叉验证，将输入的训练集拆分为训练/验证折来评估模型性能。此时手动向grid.fit()传递validation_data参数，会与GridSearchCV的内置CV逻辑冲突，导致SciKeras内部无法正确解析loss指标，触发错误。

修正步骤 #

1. 移除validation_data参数：删除grid.fit()中的validation_data=(Xvalidation, yvalidation)，由GridSearchCV自行处理交叉验证的验证集。
2. 统一Keras导入：确保所有Keras相关组件（模型、层、优化器、指标等）的导入来源一致，避免混合使用tf.keras和keras。
3. 给指标命名（可选但推荐）：为metrics中的指标指定名称，避免内部指标名称冲突。

修正后的代码 #

# 统一导入tensorflow.keras下的组件
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.losses import MeanSquaredError
from tensorflow.keras.metrics import MeanSquaredError
from scikeras.wrappers import KerasRegressor
from sklearn.model_selection import GridSearchCV

def create_lstm_model():
    model = Sequential([
        LSTM(5, input_shape=(Xtrain.shape[1], Xtrain.shape[2]), dropout=0.1, activation='tanh', return_sequences=True),
        LSTM(10, dropout=0.05, activation='tanh'),
        Dense(5, activation='relu'),
        Dense(1)
    ])
    model.compile(
        optimizer=Adam(), 
        loss=MeanSquaredError(), 
        metrics=[MeanSquaredError(name="mse")]  # 为指标指定名称
    )
    return model

model = KerasRegressor(build_fn=create_lstm_model, verbose=1)

param_grid = {
    "batch_size": [16, 32],
    "epochs": [3, 4]
}

grid = GridSearchCV(estimator=model, param_grid=param_grid, cv=3)
grid.fit(Xtrain, ytrain)  # 不再传递validation_data

# 查看调优结果
print("最佳参数:", grid.best_params_)
print("最佳交叉验证得分:", grid.best_score_)

额外说明 #

如果需要在训练过程中监控独立的验证集，可通过在create_lstm_model中添加EarlyStopping等回调函数实现，而非向grid.fit()传递validation_data。例如：

from tensorflow.keras.callbacks import EarlyStopping

def create_lstm_model():
    model = Sequential([...])  # 模型结构不变
    model.compile(...)  # 编译参数不变
    
    # 添加回调函数监控外部验证集
    callbacks = [EarlyStopping(monitor="val_mse", patience=2, restore_best_weights=True)]
    return model, callbacks  # 返回模型和回调

# 创建KerasRegressor时指定callbacks参数
model = KerasRegressor(build_fn=create_lstm_model, verbose=1, callbacks=callbacks)

内容的提问来源于stack exchange，提问作者kamil1992