TensorFlow下MobileNetV2二分类训练验证效果好但单图预测异常

问题复现详情 #

1. 数据集加载 #

使用image_dataset_from_directory加载两类存放在A、B文件夹的数据集：

BATCH_SIZE = 32
IMG_SIZE = (224, 224)
train_directory = "Train_set/"
test_directory = "Test_set/"
train_dataset = image_dataset_from_directory(train_directory, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)
validation_dataset = image_dataset_from_directory(test_directory, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)

2. 模型构建 #

使用MobileNetV2做迁移学习，内置预处理层：

preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input

def alpaca_model(image_shape=IMG_SIZE):
    input_shape = image_shape + (3,)    
    base_model = tf.keras.applications.MobileNetV2(input_shape=input_shape,
                                                   include_top=False,
                                                   weights='imagenet')
    
    # 冻结基模型
    base_model.trainable = False
    inputs = tf.keras.Input(shape=input_shape) 
    # 模型内置预处理
    x = preprocess_input(inputs) 
    x = base_model(x, training=False) 
    x = tf.keras.layers.GlobalAveragePooling2D()(x) 
    x = tf.keras.layers.Dropout(0.2)(x)
    prediction_layer = tf.keras.layers.Dense(1, activation="sigmoid")
    outputs = prediction_layer(x) 
    model = tf.keras.Model(inputs, outputs)
    return model

模型结构如下：

Model: "model_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_4 (InputLayer)        [(None, 224, 224, 3)]     0         
                                                                 
 tf.math.truediv_1 (TFOpLamb  (None, 224, 224, 3)      0         
 da)                                                             
                                                                 
 tf.math.subtract_1 (TFOpLam  (None, 224, 224, 3)      0         
 bda)                                                            
                                                                 
 mobilenetv2_1.00_224 (Funct  (None, 7, 7, 1280)       2257984   
 ional)                                                          
                                                                 
 global_average_pooling2d_1   (None, 1280)             0         
 (GlobalAveragePooling2D)                                        
                                                                 
 dropout_1 (Dropout)         (None, 1280)              0         
                                                                 
 dense_1 (Dense)             (None, 1)                 1281      
                                                                 
=================================================================
Total params: 2,259,265
Trainable params: 1,281
Non-trainable params: 2,257,984
_________________________________________________________________

3. 模型编译与训练 #

loss_function=tf.keras.losses.BinaryCrossentropy()
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
metrics=['accuracy', tf.metrics.Recall(), tf.metrics.Precision()]

训练验证指标表现优异，验证集准确率接近100%：

total_epochs = 5
history_fine = model2.fit(train_dataset, epochs=total_epochs, validation_data=validation_dataset)
Epoch 1/5
54/54 [==============================] - 213s 3s/step - loss: 0.2236 - accuracy: 0.9013 - recall: 0.9149 - precision: 0.8852 - val_loss: 0.0856 - val_accuracy: 0.9887 - val_recall: 0.9950 - val_precision: 0.9803
Epoch 2/5
54/54 [==============================] - 217s 4s/step - loss: 0.0614 - accuracy: 0.9855 - recall: 0.9928 - precision: 0.9776 - val_loss: 0.0439 - val_accuracy: 0.9977 - val_recall: 1.0000 - val_precision: 0.9950
Epoch 3/5
54/54 [==============================] - 216s 4s/step - loss: 0.0316 - accuracy: 0.9948 - recall: 0.9988 - precision: 0.9905 - val_loss: 0.0297 - val_accuracy: 0.9977 - val_recall: 1.0000 - val_precision: 0.9950
Epoch 4/5
54/54 [==============================] - 217s 4s/step - loss: 0.0258 - accuracy: 0.9954 - recall: 1.0000 - precision: 0.9905 - val_loss: 0.0373 - val_accuracy: 0.9910 - val_recall: 0.9850 - val_precision: 0.9949
Epoch 5/5
54/54 [==============================] - 220s 4s/step - loss: 0.0242 - accuracy: 0.9942 - recall: 0.9988 - precision: 0.9893 - val_loss: 0.0225 - val_accuracy: 0.9977 - val_recall: 1.0000 - val_precision: 0.9950

model2.evaluate(validation_dataset)
14/14 [==============================] - 15s 354ms/step - loss: 0.0225 - accuracy: 0.9977 - recall: 1.0000 - precision: 0.9950

4. 异常表现 #

单图预测时两类预测值没有可分边界，单图预测代码如下：

A = []
for i in os.listdir("Test_set\A"):
    location = f"Test_set\A\{i}"
    my_image = tf.keras.preprocessing.image.load_img(location, target_size=(224, 224))
    preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input

    #preprocess the image
    my_image = tf.keras.preprocessing.image.img_to_array(my_image)
    my_image = my_image.reshape((1, my_image.shape[0], my_image.shape[1], 
    my_image.shape[2]))
    my_image = preprocess_input(my_image)

    #make the prediction
    prediction = model2.predict(my_image)
    A.append(float(prediction))
B = []
for i in os.listdir("Test_set\B"):
    location = f"Test_set\B\{i}"
    my_image = tf.keras.preprocessing.image.load_img(location, target_size=(224, 224))
    preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input

    #preprocess the image
    my_image = tf.keras.preprocessing.image.img_to_array(my_image)
    my_image = my_image.reshape((1, my_image.shape[0], my_image.shape[1], 
    my_image.shape[2]))
    my_image = preprocess_input(my_image)

    #make the prediction
    prediction = model2.predict(my_image)
    B.append(float(prediction))

预测结果可视化：
#

问题原因与解决方案 #

核心原因 #

你的模型内部已经集成了preprocess_input预处理层（模型结构里的tf.math.truediv和tf.math.subtract就是对应的操作），单图预测时你又手动调用了一次preprocess_input对图像做二次归一化，导致输入模型的像素值范围不符合训练时的分布，输出结果自然混乱。

次要排查点 #

1. image_dataset_from_directory默认按文件夹名字典序分配标签，A对应0、B对应1，如果你判定类别时搞反了类标对应关系，也会误以为预测结果错误，可以先打印train_dataset.class_names确认类标顺序。
2. Windows路径下使用反斜杠\可能存在转义问题，建议统一用正斜杠/或者原始字符串r"Test_set\A"。

修复方法 #

修改单图预测逻辑，删除手动预处理的步骤即可：

# 修正后的单图预测代码
A = []
for i in os.listdir("Test_set/A"):
    location = f"Test_set/A/{i}"
    my_image = tf.keras.preprocessing.image.load_img(location, target_size=(224, 224))
    my_image = tf.keras.preprocessing.image.img_to_array(my_image)
    my_image = tf.expand_dims(my_image, axis=0)
    # 删掉手动预处理的代码：my_image = preprocess_input(my_image)
    prediction = model2.predict(my_image)
    A.append(float(prediction))
# B类的预测逻辑同理修改

修改后两类的预测值会出现明显的0、1分界，和训练验证的指标表现一致。 #

内容的提问来源于stack exchange，提问作者Chinmay Kulkarni