TensorFlow实现中遭遇拟合错误，寻求技术协助

错误原因 #

Scikeras的KerasClassifier在跨折叠序列化模型时，无法处理提前实例化的优化器对象（比如你代码中提前创建的Adagrad实例），必须在模型构建函数内部动态创建优化器实例。此外代码中还有几处细节问题会导致训练异常：

• 硬编码输入维度input_dim=10，实际数据维度可能不匹配
• EarlyStopping监控accuracy（训练集准确率）容易过拟合，应该监控验证集指标
• 激活函数列表中混合了字符串和LeakyReLU实例，需要统一处理方式

修复后的代码 #

import numpy as np
import pandas as pd
import os
os.environ["TF_USE_LEGACY_KERAS"] = 'True'
import tf_keras as keras
from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold
from sklearn.metrics import make_scorer, accuracy_score
from scikeras.wrappers import KerasClassifier
from bayes_opt import BayesianOptimization

# 初始化LeakyReLU实例
leaky_relu = keras.layers.LeakyReLU(alpha=0.1)
score_acc = make_scorer(accuracy_score)

# 数据加载与预处理
trainSet = pd.read_csv('/Users/asam/Desktop/D_Drive/ET_Prediction/layers_tune/train.csv')
train = trainSet.drop(columns=['Name', 'Ticket', 'Cabin']).dropna(axis=0)
train = pd.get_dummies(train)

X = train.drop(columns=['PassengerId','Survived'], axis=0)
y = train['Survived']
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=111, stratify=y)

# 动态获取输入维度
INPUT_DIM = X_train.shape[1]

def nn_cl_bo2(neurons, activation_idx, optimizer_idx, learning_rate, batch_size, epochs,
              layers1, layers2, normalization, dropout, dropout_rate):
    # 优化器列表：存储类而非实例
    optimizer_classes = [keras.optimizers.legacy.SGD, 
                         keras.optimizers.legacy.Adam, 
                         keras.optimizers.legacy.RMSprop, 
                         keras.optimizers.legacy.Adagrad, 
                         keras.optimizers.legacy.Ftrl]
    # 激活函数列表：区分字符串和自定义实例
    activation_options = ['relu', 'sigmoid', 'softplus', 'softsign', 'tanh', 'selu',
                         'elu', 'exponential', leaky_relu, 'relu']
    
    # 处理超参数的类型转换
    neurons = round(neurons)
    activation = activation_options[round(activation_idx)]
    optimizer_cls = optimizer_classes[round(optimizer_idx)]
    batch_size = round(batch_size)
    epochs = round(epochs)
    layers1 = round(layers1)
    layers2 = round(layers2)
    
    def nn_cl_fun():
        nn = keras.Sequential()
        # 输入层：使用动态获取的维度
        nn.add(keras.layers.Dense(neurons, input_dim=INPUT_DIM))
        # 处理激活函数：如果是实例则单独添加层
        if isinstance(activation, keras.layers.LeakyReLU):
            nn.add(activation)
        else:
            nn.layers[-1].activation = keras.activations.get(activation)
        
        # 批量归一化层
        if normalization > 0.5:
            nn.add(keras.layers.BatchNormalization())
        
        # 第一层隐藏层循环
        for _ in range(layers1):
            nn.add(keras.layers.Dense(neurons))
            if isinstance(activation, keras.layers.LeakyReLU):
                nn.add(activation)
            else:
                nn.layers[-1].activation = keras.activations.get(activation)
        
        # Dropout层
        if dropout > 0.5:
            nn.add(keras.layers.Dropout(dropout_rate, seed=123))
        
        # 第二层隐藏层循环
        for _ in range(layers2):
            nn.add(keras.layers.Dense(neurons))
            if isinstance(activation, keras.layers.LeakyReLU):
                nn.add(activation)
            else:
                nn.layers[-1].activation = keras.activations.get(activation)
        
        # 输出层
        nn.add(keras.layers.Dense(1, activation='sigmoid'))
        
        # 在模型内部创建优化器实例
        optimizer = optimizer_cls(learning_rate=learning_rate)
        nn.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
        return nn
    
    # EarlyStopping监控验证集准确率，避免过拟合
    es = keras.callbacks.EarlyStopping(monitor='val_accuracy', mode='max', verbose=0, patience=10)
    nn = KerasClassifier(model=nn_cl_fun, epochs=epochs, batch_size=batch_size, verbose=0)
    kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=123)
    # 交叉验证时传入验证集监控的回调
    score = cross_val_score(nn, X_train, y_train, scoring=score_acc, cv=kfold, 
                            fit_params={'callbacks':[es], 'validation_split':0.1}).mean()
    return score

# 超参数搜索空间
params_nn2 ={
    'neurons': (10, 100),
    'activation_idx':(0, 9),
    'optimizer_idx':(0,4),
    'learning_rate':(0.001, 0.1),  # 缩小学习率范围，原范围0.01-1过大
    'batch_size':(32, 256),        # 原范围200-1000过大，不适合小数据集
    'epochs':(20, 100),
    'layers1':(1,3),
    'layers2':(1,3),
    'normalization':(0,1),
    'dropout':(0,1),
    'dropout_rate':(0,0.3)
}

# 启动贝叶斯优化
nn_bo = BayesianOptimization(nn_cl_bo2, params_nn2, random_state=111)
nn_bo.maximize(init_points=10, n_iter=20)  # 调整迭代次数，原参数迭代太少

关键修改点说明 #

• 优化器创建方式：将优化器类存储在列表中，在模型构建函数内部用learning_rate实例化，避免序列化问题
• 输入维度动态获取：用X_train.shape[1]代替硬编码的10，适配实际数据
• 激活函数处理：区分字符串激活函数和LeakyReLU实例，实例需要单独作为层添加
• EarlyStopping优化：监控val_accuracy并添加validation_split，更合理地防止过拟合
• 超参数范围调整：缩小学习率和batch_size范围，适配泰坦尼克号小数据集的训练需求

内容的提问来源于stack exchange，提问作者Alok