超参数调优报错：KerasRegressor不支持lr参数的问题求助

问题重现 #

使用scikeras.wrappers.KerasRegressor配合GridSearchCV进行超参数调优时，触发如下报错：

ValueError: Invalid parameter lr for estimator KerasRegressor.
This issue can likely be resolved by setting this parameter in the KerasRegressor constructor:
`KerasRegressor(lr=0.1)`
Check the list of available parameters with `estimator.get_params().keys()

核心原因 #

lr（学习率）是Keras优化器的参数，而非KerasRegressor本身的顶层参数。直接在param_grid中添加lr，GridSearchCV会尝试将其赋值给KerasRegressor实例，导致参数不匹配。

解决方案 #

根据调参需求，分两种场景处理：

场景1：同时调优优化器类型和对应学习率 #

通过Scikit-learn的双下划线参数语法（optimizer__lr），将学习率传递给嵌套的优化器对象。

步骤1：修改模型构建函数

让create_autoencoder接收optimizer参数，并处理字符串类型的优化器输入：

import tensorflow as tf

def create_autoencoder(optimizer="adam"):
    # 将优化器字符串转为带参数的实例
    if isinstance(optimizer, str):
        if optimizer == "adam":
            optimizer = tf.keras.optimizers.Adam()
        elif optimizer == "sgd":
            optimizer = tf.keras.optimizers.SGD()
    
    # 构建自编码器模型（示例结构，根据你的数据调整）
    input_img = tf.keras.Input(shape=(28, 28, 1))
    x = tf.keras.layers.Conv2D(32, (3,3), activation='relu', padding='same')(input_img)
    x = tf.keras.layers.MaxPool2D((2,2), padding='same')(x)
    x = tf.keras.layers.Conv2D(16, (3,3), activation='relu', padding='same')(x)
    encoded = tf.keras.layers.MaxPool2D((2,2), padding='same')(x)
    
    x = tf.keras.layers.Conv2D(16, (3,3), activation='relu', padding='same')(encoded)
    x = tf.keras.layers.UpSampling2D((2,2))(x)
    x = tf.keras.layers.Conv2D(32, (3,3), activation='relu', padding='same')(x)
    x = tf.keras.layers.UpSampling2D((2,2))(x)
    decoded = tf.keras.layers.Conv2D(1, (3,3), activation='sigmoid', padding='same')(x)
    
    autoencoder = tf.keras.Model(input_img, decoded)
    autoencoder.compile(optimizer=optimizer, loss='mse')
    return autoencoder

步骤2：调整参数网格

使用optimizer__lr传递学习率：

from scikeras.wrappers import KerasRegressor
from sklearn.model_selection import GridSearchCV

# 定义搜索参数
batch_sizes = [16, 32]
epochs_list = [10, 20]
optimizers = ["adam", "sgd"]
learning_rates = [0.001, 0.01]

param_grid = {
    "batch_size": batch_sizes,
    "epochs": epochs_list,
    "optimizer": optimizers,
    "optimizer__lr": learning_rates  # 关键：通过双下划线传递优化器参数
}

# 初始化模型并执行网格搜索
model = KerasRegressor(build_fn=create_autoencoder, verbose=0)
grid = GridSearchCV(estimator=model, n_jobs=1, verbose=10, cv=2, param_grid=param_grid)
grid_result = grid.fit(train_x_noise, train_x)

# 输出结果
best_params = grid_result.best_params_
best_score = grid_result.best_score_
print(f'Best params: {best_params}')
print(f'Best score: {best_score}')

for mean, stdev, param in zip(grid_result.cv_results_['mean_test_score'], 
                              grid_result.cv_results_['std_test_score'], 
                              grid_result.cv_results_['params']):
    print("%f (%f) with: %r" % (mean, stdev, param))

场景2：固定优化器，仅调优学习率 #

将lr作为模型构建函数的参数，由KerasRegressor传递给build_fn。

步骤1：修改模型构建函数

import tensorflow as tf

def create_autoencoder(lr=0.001):
    # 固定使用Adam优化器，接收lr参数
    optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
    
    # 模型构建同上...
    autoencoder.compile(optimizer=optimizer, loss='mse')
    return autoencoder

步骤2：调整参数网格

直接在param_grid中使用lr：

param_grid = {
    "batch_size": batch_sizes,
    "epochs": epochs_list,
    "lr": learning_rates  # lr作为build_fn的参数，合法有效
}

验证方法 #

可以先打印KerasRegressor的可用参数，确认参数是否合法：

model = KerasRegressor(build_fn=create_autoencoder, verbose=0)
print(model.get_params().keys())

内容的提问来源于stack exchange，提问作者OpMartix