为何基于Fashion MNIST训练的CNN模型对外部灰度图像分类效果极差？

问题描述 #

基于Fashion MNIST数据集用Keras-TensorFlow训练CNN模型，训练准确率96%、验证准确率91%，但使用谷歌获取的同类灰度服装图像预测时分类完全错误（如衬衫被识别为裤子、连衣裙被识别为鞋子），已对测试图像做了归一化、28×28像素转换及灰度化预处理。

原实现代码 #

import tensorflow as tf
from tensorflow import keras

(X_train, y_train), (X_valid, y_valid) = keras.datasets.fashion_mnist.load_data()

# Reshape the input data to a 4D tensor 
X_train = X_train.reshape((X_train.shape[0], 28, 28, 1))
X_valid = X_valid.reshape((X_valid.shape[0], 28, 28, 1))

# Normalize the input data to have values between 0 and 1
X_train = X_train / 255.0
X_valid = X_valid / 255.0

# myCallback
class myCallback(tf.keras.callbacks.Callback):
    # Define the method that checks the accuracy at the end of each epoch
    def on_epoch_end(self, epoch, logs={}):
            if logs.get('accuracy') >= 0.995:                 
                print("\nReached 99.5% accuracy so cancelling training!")
                # Stop training once the above condition is met
                self.model.stop_training = True
    pass

from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Define the ImageDataGenerator for data augmentation
datagen = ImageDataGenerator(
    rotation_range=10,
    width_shift_range=0.1,
    height_shift_range=0.1,
    shear_range=0.1,
    zoom_range=0.1,
    horizontal_flip=True,
    fill_mode='nearest'
)

# Flow the training and validation images in batches of 128
train_flow = datagen.flow(X_train, y_train, batch_size=128)
valid_flow = datagen.flow(X_valid, y_valid, batch_size=128)

# convolutional_model
def model():
    # Define the model
    model = tf.keras.models.Sequential([
        keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
        keras.layers.MaxPooling2D((2, 2)),
        keras.layers.Conv2D(64, (3, 3), activation='relu'),
        keras.layers.MaxPooling2D((2, 2)),
        keras.layers.Flatten(),
        keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dense(10, activation='softmax')
    ])

    # Compile the model
    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['accuracy'])
        
    return model

# Instantiate the callback class
callbacks = myCallback()

# Train the model
history = model().fit(X_train, y_train, epochs = 10, validation_data=(X_valid, y_valid), callbacks=[callbacks])

model().save('myModelName.h5')

import numpy as np
from google.colab import files
from tensorflow.keras.utils import load_img, img_to_array
from tensorflow.keras.models import load_model
import matplotlib.pyplot as pyplot

# Load the trained model
model = load_model('myModelName.h5')

# Define class labels
class_labels = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
                'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

# Prompt user to upload an image
uploaded = files.upload()

for fn in uploaded.keys():
    # Load and preprocess the image
    path = '/content/' + fn
    img = load_img(path, target_size=(28, 28), color_mode='grayscale')
    x = img_to_array(img)
    x = x.reshape((1,) + x.shape)
    x = x.astype('float32') / 255

    # Make prediction
    prediction = model.predict(x)
    predicted_class = class_labels[np.argmax(prediction)]

    # Plot the image 
    pyplot.imshow(x[0], cmap='gray')
    pyplot.show()
    print(x.shape)

    # Print the predicted class
    print(f'The uploaded image is a {predicted_class}')

核心问题排查 #

1. 致命错误：保存的是未训练的空白模型
   训练时用model().fit()创建临时模型实例并训练，但保存时又调用model().save()重新生成了一个全新的、未训练的模型，最终保存的模型根本没有学习到任何特征，这是预测完全失效的主要原因。

2. 数据增强未实际应用
   定义了ImageDataGenerator并生成了train_flow和valid_flow，但训练时仍直接使用原始的X_train和y_valid，数据增强完全没发挥作用，模型对真实世界图像的鲁棒性不足。

3. 外部图像与训练集的风格差异
   Fashion MNIST的图像是黑底白服装（背景为黑色，服装为白色），而谷歌获取的图像通常是白底黑服装，特征分布完全相反，模型无法识别。另外，外部图像可能存在多余背景、服装未居中，缩放后特征偏移。

解决方案 #

1. 修正模型训练与保存逻辑 #

# 实例化模型并训练（使用数据增强后的数据流）
model_instance = model()
history = model_instance.fit(
    train_flow,
    epochs=10,
    validation_data=valid_flow,
    callbacks=[callbacks]
)

# 保存训练完成的模型实例
model_instance.save('myModelName.h5')

2. 统一外部图像与训练集的风格 #

在预测预处理步骤中增加颜色反转（如果外部图像是白底）：

# 加载并预处理图像
path = '/content/' + fn
img = load_img(path, target_size=(28, 28), color_mode='grayscale')
x = img_to_array(img)
x = x.reshape((1,) + x.shape)
x = x.astype('float32') / 255.0

# 反转颜色，匹配Fashion MNIST的黑底风格
x = 1.0 - x

3. 优化外部图像预处理（可选） #

• 对外部图像先裁剪，保留服装主体区域后再缩放到28×28
• 增加高斯模糊等预处理，减少噪声对模型的干扰

4. 模型微调（可选） #

收集少量外部服装图像，标注后对训练好的模型进行微调，缩小训练集与真实数据的分布差异。

内容的提问来源于stack exchange，提问作者karak87rt0