Kaggle竞赛中5个EfficientNet搭建的Stacking模型效果差于基模型原因咨询
2026-10-2
问题描述
我为参与Kaggle竞赛，使用5个EfficientNet模型搭建了一个Stacking模型，该模型的架构如下：
Model: "model" __________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== input_1_0 (InputLayer) [(None, 600, 600, 3) 0 __________________________________________________________________________________________________ input_3_1 (InputLayer) [(None, 600, 600, 3) 0 __________________________________________________________________________________________________ input_5_2 (InputLayer) [(None, 600, 600, 3) 0 __________________________________________________________________________________________________ input_7_3 (InputLayer) [(None, 600, 600, 3) 0 __________________________________________________________________________________________________ input_9_4 (InputLayer) [(None, 600, 600, 3) 0 __________________________________________________________________________________________________ effnet_layer0_0 (Functional) (None, None, None, 2 64097680 input_1_0[0][0] __________________________________________________________________________________________________ effnet_layer1_1 (Functional) (None, None, None, 2 64097680 input_3_1[0][0] __________________________________________________________________________________________________ effnet_layer2_2 (Functional) (None, None, None, 2 64097680 input_5_2[0][0] __________________________________________________________________________________________________ effnet_layer3_3 (Functional) (None, None, None, 2 64097680 input_7_3[0][0] __________________________________________________________________________________________________ effnet_layer4_4 (Functional) (None, None, None, 2 64097680 input_9_4[0][0] __________________________________________________________________________________________________ global_average_pooling2d_0 (Glo (None, 2560) 0 effnet_layer0_0[0][0] __________________________________________________________________________________________________ global_average_pooling2d_1_1 (G (None, 2560) 0 effnet_layer1_1[0][0] __________________________________________________________________________________________________ global_average_pooling2d_2_2 (G (None, 2560) 0 effnet_layer2_2[0][0] __________________________________________________________________________________________________ global_average_pooling2d_3_3 (G (None, 2560) 0 effnet_layer3_3[0][0] __________________________________________________________________________________________________ global_average_pooling2d_4_4 (G (None, 2560) 0 effnet_layer4_4[0][0] __________________________________________________________________________________________________ dropout_0 (Dropout) (None, 2560) 0 global_average_pooling2d_0[0][0] __________________________________________________________________________________________________ dropout_1_1 (Dropout) (None, 2560) 0 global_average_pooling2d_1_1[0][0 __________________________________________________________________________________________________ dropout_2_2 (Dropout) (None, 2560) 0 global_average_pooling2d_2_2[0][0 __________________________________________________________________________________________________ dropout_3_3 (Dropout) (None, 2560) 0 global_average_pooling2d_3_3[0][0 __________________________________________________________________________________________________ dropout_4_4 (Dropout) (None, 2560) 0 global_average_pooling2d_4_4[0][0 __________________________________________________________________________________________________ dense_0 (Dense) (None, 4) 10244 dropout_0[0][0] __________________________________________________________________________________________________ dense_1_1 (Dense) (None, 4) 10244 dropout_1_1[0][0] __________________________________________________________________________________________________ dense_2_2 (Dense) (None, 4) 10244 dropout_2_2[0][0] __________________________________________________________________________________________________ dense_3_3 (Dense) (None, 4) 10244 dropout_3_3[0][0] __________________________________________________________________________________________________ dense_4_4 (Dense) (None, 4) 10244 dropout_4_4[0][0] __________________________________________________________________________________________________ concatenate (Concatenate) (None, 20) 0 dense_0[0][0] dense_1_1[0][0] dense_2_2[0][0] dense_3_3[0][0] dense_4_4[0][0] __________________________________________________________________________________________________ dense (Dense) (None, 10) 210 concatenate[0][0] __________________________________________________________________________________________________ dense_1 (Dense) (None, 4) 44 dense[0][0] ================================================================================================== Total params: 320,539,874 Trainable params: 254 Non-trainable params: 320,539,620
Stacking模型的性能指标如下：
单个基模型的性能指标如下：
当前问题：使用该Stacking模型生成Kaggle竞赛的预测结果时，得分仅为0.551，而使用任意单个基模型时得分可达0.581。Stacking模型不是理应得到优于基模型的结果吗？
问题原因分析
Stacking效果优于基模型的核心前提是基模型误差具备互补性、元特征构建无泄露，你的场景效果下降大概率是以下原因导致的：
- 基模型多样性严重不足
5个基模型均为同结构的EfficientNet，从参数数量看甚至权重都没有差异，模型的错误模式高度重叠，元学习器无法从高度相关的基模型输出中学习到更优的融合规则，反而会引入冗余噪声拉低效果。 - 元学习器训练存在数据泄露
你的基模型权重全部冻结，若你直接用基模型在完整训练集上的预测结果作为元特征训练顶层的全连接层，会导致严重的过拟合：基模型已经见过所有训练样本，输出的预测结果置信度偏高，元学习器会过拟合该分布，在未见过的测试集上效果暴跌。正确的Stacking需要用K折交叉验证生成基模型的离褶（Out-of-Fold）预测结果作为元特征训练元学习器，避免数据泄露。 - 元学习器容量或训练策略不合理
你当前使用的两层全连接层元学习器容量偏低，同时没有加入正则约束，要么无法学习到有效融合规则，要么在相关性极高的基模型输出上出现过拟合。建议先尝试更简单的融合方案（比如加权平均、软投票）验证融合效果上限，再调整元学习器结构。 - 本地验证和线上分布不匹配
你本地验证集的Stacking指标优于基模型，但线上效果相反，说明你的本地验证集划分没有对齐Kaggle测试集的分布，比如没有按样本分组、没有模拟分布偏移，导致你选出的模型checkpoint是过拟合本地验证集的。
内容的提问来源于stack exchange，提问作者Arthur
相关产品推荐
相关产品推荐
