使用GridSearchCV调优MLPRegressor：批量设置hidden_layer_sizes方法

问题背景 #

使用GridSearchCV对MLPRegressor进行超参数调优时，hidden_layer_sizes要求传入元组类型参数。希望无需手动输入，批量生成多种隐藏层尺寸组合，但复用SVR中np.arange生成参数的逻辑时失败。

现有可运行代码 #

param_list = {"hidden_layer_sizes": [(200,70), (150,100,50), (120,80,40), (100,50,30)], "activation": ["identity", "logistic", "tanh", "relu"], "solver": ["lbfgs", "sgd", "adam"], "alpha": [0.0001,0.0005]}
MLP_gridCV = GridSearchCV(
    estimator=MLPRegressor(max_iter=10000, n_iter_no_change=30),
    param_grid=param_list,
    n_jobs=-1,
    cv=3,
    verbose=5,
)
MLP_gridCV.fit(X_train, y_train.ravel())

# 预测
y_pred8 = MLP_gridCV.predict(X_test)

此前SVR参数生成方式（可正常运行） #

gamma = np.arange(0.001, 0.1, 0.001).tolist()

尝试的无效代码 #

learning_rate_init= [0.0001,0.0002]
first_layer_neurons= np.arange(10, 200, 10).tolist()
second_layer_neurons= np.arange(10, 200, 10).tolist()
hidden_layer_sizes = [first_layer_neurons,second_layer_neurons]
activation= ['identity', 'tanh', 'relu']
   
params_grid = {
                'hidden_layer_sizes':hidden_layer_sizes,
                'learning_rate_init':learning_rate_init,
                'activation':activation}

print(params_grid)
# 输出：
# {'hidden_layer_sizes': [[10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190],
# [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190]],
# 'learning_rate_init': [0.0001, 0.0002], 
# 'activation': ['identity', 'tanh', 'relu']}

解决方案 #

核心问题是：hidden_layer_sizes需要的是元素为元组的列表（如[(10,20), (30,40)]），而非嵌套列表。可以用itertools.product生成所有可能的神经元组合，再转换为元组。

方法1：生成两层隐藏层的所有组合 #

如果仅需测试两层隐藏层的所有搭配，用笛卡尔积生成组合：

import numpy as np
from itertools import product

learning_rate_init = [0.0001, 0.0002]
first_layer_neurons = np.arange(10, 200, 10).tolist()
second_layer_neurons = np.arange(10, 200, 10).tolist()

# 生成所有两层组合，转为元组
hidden_layer_sizes = list(product(first_layer_neurons, second_layer_neurons))
# 可选：添加单层隐藏层的情况
hidden_layer_sizes += [(n,) for n in first_layer_neurons]

activation = ['identity', 'tanh', 'relu']

params_grid = {
    'hidden_layer_sizes': hidden_layer_sizes,
    'learning_rate_init': learning_rate_init,
    'activation': activation
}

方法2：生成多层数（1/2/3层）的组合 #

如果要测试不同层数的隐藏层结构，可分别生成后合并：

import numpy as np
from itertools import product

neuron_options = np.arange(10, 200, 10).tolist()

hidden_layer_sizes = []
# 1层隐藏层
hidden_layer_sizes += [(n,) for n in neuron_options]
# 2层隐藏层
hidden_layer_sizes += list(product(neuron_options, neuron_options))
# 3层隐藏层
hidden_layer_sizes += list(product(neuron_options, neuron_options, neuron_options))

# 其他参数
learning_rate_init = [0.0001, 0.0002]
activation = ['identity', 'tanh', 'relu']

params_grid = {
    'hidden_layer_sizes': hidden_layer_sizes,
    'learning_rate_init': learning_rate_init,
    'activation': activation
}

无效代码原因分析 #

之前的代码将hidden_layer_sizes设为嵌套列表[first_layer_neurons, second_layer_neurons]，GridSearchCV会把整个列表当作单个参数值传入，而非逐个测试元组结构，不符合MLPRegressor对该参数的要求。

内容的提问来源于stack exchange，提问作者ahm ane