VGG16花卉分类CNN训练验证准确率76%，手动预测仅6%问题求助

问题现象 #

• 基于VGG16搭建16类花卉分类CNN，数据集每类含70张训练图、10张验证/测试图
• 训练100轮后训练准确率约99%、验证准确率约76%，model.evaluate()测试测试集准确率超70%
• 手动预测测试集并对比标签计算准确率仅约6.25%，接近16类随机猜测的概率

相关代码 #

模型训练代码 #

#import all the necessary libraries and be specific so as to avoid wasting time importing everything 
import sys
import matplotlib.pyplot as plt
import numpy as np
from tensorflow.keras.applications import VGG16
from tensorflow.keras import optimizers
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
import sklearn.metrics as metrics
from sklearn.metrics import confusion_matrix
import seaborn as sns  
model_type = 'vgg16'

# Loading the VGG Model
vgg_model = VGG16(weights='imagenet', include_top=False, input_shape=(200,200,3))
vgg_model.trainable = False
model = tf.keras.Sequential([vgg_model,
                                 tf.keras.layers.GlobalAveragePooling2D(),
                                 tf.keras.layers.Dropout(0.1),
                                 tf.keras.layers.Dense(512, activation= "relu"),
                                 tf.keras.layers.BatchNormalization(),
                                 tf.keras.layers.Dropout(0.1),
                                 tf.keras.layers.Dense(16, activation="softmax")                                     
                                ])

datagen = ImageDataGenerator(featurewise_center=True)
#retrieve the trian data using imagedatagen 
train = datagen.flow_from_directory('/content/16_flowers/Train/',
  class_mode='categorical', color_mode= 'rgb',batch_size=64, target_size=(200, 200))
test = datagen.flow_from_directory('/content/16_flowers/Test/',
  class_mode='categorical', color_mode= 'rgb',batch_size=10, target_size=(200, 200))

base_learning_rate = 0.00005
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learning_rate),loss = tf.keras.metrics.categorical_crossentropy,metrics=['accuracy'])

history = model.fit(train,epochs = 100 , validation_data = test)

#summarise_diagnostics(history)
model.save("vgg16CNN.model")

_, acc = model.evaluate(test, steps=len(test), verbose=0)
print('> %.3f' % (acc * 100.0))

plot_cm(model)

手动预测及准确率计算代码 #

datagen = ImageDataGenerator(featurewise_center=True)
test = datagen.flow_from_directory('/content/16_flowers/Test/',
  class_mode='categorical', color_mode= 'rgb',batch_size=10, target_size=(200, 200))

predictions = model.predict(test)
predicted_classes = np.argmax(predictions, axis=1)

true_classes = test.classes
count = 0
for i in range(len(predicted_classes)):
  print(true_classes[i],predicted_classes[i])
  if true_classes[i] == predicted_classes[i]:
    count +=1
print(count/160*100)

问题成因分析 #

• 样本顺序不匹配是核心原因：flow_from_directory默认开启shuffle=True，调用model.predict()时生成器会随机打乱样本输出顺序，但test.classes是按照文件夹排序的固定标签列表，二者顺序完全不对应，最终计算的准确率就和随机猜测一致，刚好16类对应6.25%的准确率。修复方法是创建测试集生成器时指定shuffle=False，保证预测样本顺序和标签顺序一致。
• 预处理参数未对齐：代码中使用featurewise_center=True时，必须先调用datagen.fit(训练集样本)计算训练集的均值，再做标准化处理。你训练阶段没有做fit操作，手动预测阶段新建的ImageDataGenerator也没有复用训练时的统计参数，预处理逻辑不一致也会导致预测结果出错。
• 模型加载异常：如果是跨脚本加载保存的模型做预测，需要确认保存的模型权重、配置完全正确，自定义的损失函数、层结构需要在加载时正确注册，否则会出现模型推理结果完全错误的问题。
• 标签编码不匹配：确认训练集和测试集生成器的class_indices编码规则一致，同一目录结构下该问题出现概率较低，可作为排查兜底项。

内容的提问来源于stack exchange，提问作者danCodes