自研Keras多分类模型无性能提升问题排查求助

问题描述 #

用Keras搭建了与VGG同结构的多分类模型，但训练后准确率始终固定在0.3212，验证准确率0.3227，完全没有性能提升。替换为预训练VGG模型后效果良好，用PyTorch实现相同结构的模型也能正常收敛提升性能，已确认数据生成器等其他模块无问题（在VGG模型上验证通过）。

原始代码 #

import pandas as pd
import numpy as np
import os
import cv2
import matplotlib.pyplot as plt
import warnings
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from mpl_toolkits.axes_grid1 import ImageGrid


path = '/content/train/'
for folder in os.listdir(path):
    print(len(os.listdir(path + folder))) 


from keras.layers import Activation, Dropout, Flatten, Dense
def createModel():
  model = keras.models.Sequential()
  model.add(keras.layers.Conv2D(64, (3, 3), input_shape=(224, 224, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(64, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.MaxPooling2D())


  model.add(keras.layers.Conv2D(128, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(128, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.MaxPooling2D())
    
    
  model.add(keras.layers.Conv2D(256, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(256, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(256, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.MaxPooling2D())
    
    
  model.add(keras.layers.Conv2D(512, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(512, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(512, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.MaxPooling2D())
    
    
  model.add(keras.layers.Conv2D(512, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(512, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(512, (3, 3), padding='same'))
  model.add(Activation('relu'))
  model.add(keras.layers.MaxPooling2D())
    
  model.add(keras.layers.Flatten())
    
  model.add(keras.layers.Dense(4096, activation='relu'))
  model.add(keras.layers.Dense(4096, activation='relu'))
  model.add(keras.layers.Dense(5, activation ='softmax'))

  return model


# 数据增强
train_image_generator = ImageDataGenerator(
    rescale=1./255,
    rotation_range=5,  # 正负5度旋转
    validation_split=0.1)  # 10%数据作为验证集

# 训练集与验证集划分
train_dataset = train_image_generator.flow_from_directory(batch_size=32,
                                             directory='/content/train',
                                             shuffle=True,
                                             target_size=(224, 224),
                                             subset="training",
                                             class_mode='categorical')

validation_dataset = train_image_generator.flow_from_directory(batch_size=32,
                                             directory='/content/train',
                                             shuffle=True,
                                             target_size=(224, 224),
                                             subset="validation",
                                             class_mode='categorical')

# 测试集预处理
test_image_generator = ImageDataGenerator(rescale=1./255)
test_dataset = test_image_generator.flow_from_directory(
                                             directory='/content/test',
                                             shuffle=False,
                                             target_size=(224, 224),
                                             class_mode=None)


# 测试Adam与RMSprop优化器
model_adam = createModel()
model_RMSprop = createModel()

model_adam.compile(optimizer='adam', loss = 'categorical_crossentropy', metrics=['acc'])
model_RMSprop.compile(optimizer='rmsprop', loss = 'categorical_crossentropy',metrics=['acc'])
history_adam = model_adam.fit(train_dataset, epochs=60, validation_data=validation_dataset)

排查方向与解决方法 #

1. 优化权重初始化 #

VGG属于深层网络，Keras默认的权重初始化可能无法适配深层网络的梯度流动，易引发梯度消失或爆炸。建议显式使用he_normal初始化，该初始化更适配ReLU激活的网络：

# 在Conv2D和Dense层添加kernel_initializer参数
keras.layers.Conv2D(64, (3, 3), padding='same', kernel_initializer='he_normal')
keras.layers.Dense(4096, activation='relu', kernel_initializer='he_normal')

2. 添加Dropout与BatchNormalization #

原版VGG在全连接层后加入Dropout防止过拟合，BatchNormalization则能稳定训练过程、缓解梯度消失问题：

def createModel():
  model = keras.models.Sequential()
  model.add(keras.layers.Conv2D(64, (3, 3), input_shape=(224, 224, 3), padding='same', kernel_initializer='he_normal'))
  model.add(keras.layers.BatchNormalization())  # 添加BN层
  model.add(Activation('relu'))
  model.add(keras.layers.Conv2D(64, (3, 3), padding='same', kernel_initializer='he_normal'))
  model.add(keras.layers.BatchNormalization())
  model.add(Activation('relu'))
  model.add(keras.layers.MaxPooling2D())

  # 后续卷积层重复添加BN与初始化参数...
    
  model.add(keras.layers.Flatten())
    
  model.add(keras.layers.Dense(4096, activation='relu', kernel_initializer='he_normal'))
  model.add(keras.layers.Dropout(0.5))  # 添加Dropout
  model.add(keras.layers.Dense(4096, activation='relu', kernel_initializer='he_normal'))
  model.add(keras.layers.Dropout(0.5))
  model.add(keras.layers.Dense(5, activation ='softmax'))

  return model

3. 调整学习率 #

Adam默认学习率0.001对于从头训练的深层网络可能过大，导致模型无法收敛。尝试降低学习率：

model_adam.compile(optimizer=keras.optimizers.Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['acc'])

4. 对齐输入预处理 #

预训练VGG使用了特定的输入预处理（如减去ImageNet均值），可以尝试对齐该预处理逻辑：

# 修改数据生成器的预处理方式
train_image_generator = ImageDataGenerator(
    preprocessing_function=tf.keras.applications.vgg16.preprocess_input,
    rotation_range=5,
    validation_split=0.1)

test_image_generator = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input)

5. 监控梯度流动 #

可以在训练过程中监控梯度值，确认是否存在梯度消失问题：

# 添加梯度监控回调
class GradientMonitor(tf.keras.callbacks.Callback):
    def on_train_batch_end(self, batch, logs=None):
        grads = [tf.norm(g) for g in model_adam.trainable_variables]
        print(f"Batch {batch} - 平均梯度范数: {np.mean(grads)}")

history_adam = model_adam.fit(train_dataset, epochs=60, validation_data=validation_dataset, callbacks=[GradientMonitor()])

如果梯度均值趋近于0，说明存在梯度消失问题，需要进一步优化网络结构（如增加BN层、调整初始化方式）。

内容的提问来源于stack exchange，提问作者Ouismed