使用遗传算法优化Keras ANN模型超参数时遇参数错误求助

问题场景与代码 #

我尝试用sklearn_genetic的GASearchCV优化Keras模型的超参数，代码如下：

import tensorflow as tf
from tensorflow import keras
from scikeras.wrappers import KerasClassifier
from sklearn_genetic import GASearchCV
from sklearn_genetic.space import Categorical, Integer

def ann_model_ga(number_of_hidden_layer=1,
                 number_of_neurons=50,
                 optimizer="adam"):
    model = keras.models.Sequential()
    model.add(keras.layers.Flatten(input_shape=X_train.shape[1:]))
    for hidden_layer in range(number_of_hidden_layer):
        model.add(keras.layers.Dense(number_of_neurons))
    model.add(keras.layers.Dense(10, activation="softmax"))
    model.compile(optimizer=optimizer, loss="categorical_crossentropy", metrics=["accuracy", "AUC"],)
    return model

param_grid = {'number_of_hidden_layer': Integer(1, 5),
              'number_of_neurons': Integer(100, 200),
              'optimizer': Categorical(["adam", "sgd"])}

estimator = KerasClassifier(build_fn=ann_model_ga)

evolved_estimator = GASearchCV(estimator=estimator,
                               cv=5,
                               scoring='accuracy',
                               population_size=10,
                               generations=35,
                               tournament_size=3,
                               elitism=True,
                               crossover_probability=0.8,
                               mutation_probability=0.1,
                               param_grid=param_grid,
                               criteria='max',
                               algorithm='eaMuPlusLambda',
                               n_jobs=-1,
                               verbose=True,
                               keep_top_k=4)
history_ga = evolved_estimator.fit(X_train, y_train)

报错信息 #

运行后出现如下错误：

ValueError                                Traceback (most recent call last)
<ipython-input-17-36285fd9a509> in <cell line: 1>()
----> 1 history_ga = evolved_estimator.fit(X_train, y_train)

4 frames
/usr/local/lib/python3.10/dist-packages/scikeras/wrappers.py in set_params(self, **params)
   1163                     # Give a SciKeras specific user message to aid
   1164                     # in moving from the Keras wrappers
-> 1165                     raise ValueError(
   1166                         f"Invalid parameter {param} for estimator {self.__name__}."
   1167                         "\nThis issue can likely be resolved by setting this parameter"

ValueError: Invalid parameter number_of_hidden_layer for estimator KerasClassifier.
This issue can likely be resolved by setting this parameter in the KerasClassifier constructor:
`KerasClassifier(number_of_hidden_layer=1)`
Check the list of available parameters with `estimator.get_params().keys()`

解决方案 #

原因分析 #

SciKeras的KerasClassifier不会自动识别build_fn中的参数，必须明确告知这些参数属于模型构建函数，否则GASearchCV会把它们当作KerasClassifier自身的参数，从而报错。

方法一：在KerasClassifier初始化时声明模型参数 #

把模型构建函数的参数作为关键字参数传入KerasClassifier的构造器，让这些参数成为估计器的可配置参数：

# 修改estimator的初始化
estimator = KerasClassifier(build_fn=ann_model_ga,
                            number_of_hidden_layer=1,
                            number_of_neurons=50,
                            optimizer="adam")

此时原有的param_grid无需修改，GASearchCV可以正常识别并优化这些参数。

方法二：在参数网格中使用build_fn__前缀 #

通过添加前缀build_fn__，明确指定这些参数是传递给build_fn的：

# 修改param_grid
param_grid = {'build_fn__number_of_hidden_layer': Integer(1, 5),
              'build_fn__number_of_neurons': Integer(100, 200),
              'build_fn__optimizer': Categorical(["adam", "sgd"])}

这种方式不需要修改KerasClassifier的初始化代码，直接通过前缀完成参数的路由。

额外优化：避免依赖全局变量 #

原模型函数中直接使用了全局变量X_train来定义输入形状，建议将input_shape作为参数传入模型函数，提升代码封装性：

def ann_model_ga(number_of_hidden_layer=1,
                 number_of_neurons=50,
                 optimizer="adam",
                 input_shape=None):
    model = keras.models.Sequential()
    model.add(keras.layers.Flatten(input_shape=input_shape))
    for hidden_layer in range(number_of_hidden_layer):
        model.add(keras.layers.Dense(number_of_neurons))
    model.add(keras.layers.Dense(10, activation="softmax"))
    model.compile(optimizer=optimizer, loss="categorical_crossentropy", metrics=["accuracy", "AUC"],)
    return model

# 初始化estimator时传入input_shape
estimator = KerasClassifier(build_fn=ann_model_ga,
                            input_shape=X_train.shape[1:])

内容的提问来源于stack exchange，提问作者Zahra Reyhanian