构建唇语单词分类预测模型时遭遇矩阵尺寸不兼容错误求助

问题描述 #

构建基于唇部图像的单词分类模型，使用含142657张预处理图像的单说话者副词数据集，训练时未完成第一个epoch即出现矩阵尺寸不兼容错误。

代码实现 #

import os
from silence_tensorflow import silence_tensorflow

silence_tensorflow()
import tensorflow as tf
from tensorflow.keras.layers import Dense, Activation, Dropout, Input, Conv2D, \
MaxPooling2D, Flatten, BatchNormalization
from tensorflow.keras.models import Sequential
from tensorflow.keras.preprocessing.image import ImageDataGenerator

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
tf.autograph.set_verbosity(0)
tf.get_logger().setLevel('ERROR')


class AdverbNet(object):
    def __init__(self):
        self.Model = Sequential()
        self.build()

    def build(self):
        self.Model.add(Input(name='the_input', shape=(224, 224, 1), batch_size=16, dtype='float32'))
        self.Model.add(Conv2D(32, (3, 3), activation='sigmoid', name='convo2'))
        self.Model.add(MaxPooling2D(pool_size=(2, 2)))
        self.Model.add(Conv2D(32, (3, 3), activation='sigmoid', name='convo3'))
        self.Model.add(MaxPooling2D(pool_size=(2, 2)))
        self.Model.add(Conv2D(64, (3, 3), activation='relu', name='convo4'))
        self.Model.add(MaxPooling2D(pool_size=(2, 2)))
        self.Model.add(Flatten())

        self.Model.add(Dense(512))
        self.Model.add(Dropout(0.5))
        self.Model.add(BatchNormalization(scale=False))
        self.Model.add(Activation('relu'))
        self.Model.add(Dropout(0.5))
        self.Model.add(Dense(4, activation='softmax'))

    def summary(self):
        self.Model.summary()


if __name__ == "__main__":
    common_path = 'C:/Users/Loide/Desktop/Liphy/'
    C = AdverbNet()
    C.Model.compile(optimizer="Adam", loss='categorical_crossentropy', metrics=['accuracy'])
    C.Model.summary()

with tf.device('/device:GPU:0'):
    batch_size = 16
    epochs = 32
    train_dir = common_path + 'Images/Adverb/'
    test_dir = common_path + 'Images/Adverb/'
    checkpoint_path = common_path + 'SavedModels/Adverb/'

train_image_generator = ImageDataGenerator(rescale=1. / 255)  # Generator for training data generate training anD test set
train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,
    directory=train_dir,
    shuffle=True,
    target_size=(224, 224),
    class_mode='categorical',
    color_mode='grayscale')
    

test_image_generator = ImageDataGenerator(rescale=1. / 255)  # Generator for test data
test_data_gen = test_image_generator.flow_from_directory(batch_size=batch_size,
    directory=test_dir,
    shuffle=False,
    target_size=(224, 224),
    class_mode='categorical',
    color_mode='grayscale')

callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy',
                                                patience=10,
                                                restore_best_weights=True,
                                                baseline=0.45)
history = C.Model.fit(train_data_gen,
                          steps_per_epoch=8916,  # Number of images // Batch size
    epochs=epochs,
    verbose=1,
    validation_data=test_data_gen,
    validation_steps=187,
    callbacks=[callback])

C.Model.save(checkpoint_path, save_format='tf')

错误信息 #

[Matrix size-incompatible: In[0]: [1,43264], In[1]: [16,512]
         [[{{node gradient_tape/sequential/dense/MatMul/MatMul_1}}]] [Op:__inference_train_function_1179]

问题分析 #

错误根源是Input层固定了batch_size=16，导致模型被构建为仅能处理批量大小为16的输入。但训练数据集总样本数142657无法被16整除（142657 = 16×8916 + 1），最后一个训练批次仅包含1张图像，此时模型期望输入批量大小为16，但实际输入为1，导致全连接层的矩阵乘法维度不匹配（输入张量形状为[1,43264]，而权重张量形状为[16,512]）。

解决方案 #

移除Input层中的batch_size参数，让模型支持任意批量大小的输入：

修改AdverbNet类的build方法：

def build(self):
    # 移除batch_size参数，保留shape和dtype
    self.Model.add(Input(name='the_input', shape=(224, 224, 1), dtype='float32'))
    self.Model.add(Conv2D(32, (3, 3), activation='sigmoid', name='convo2'))
    self.Model.add(MaxPooling2D(pool_size=(2, 2)))
    self.Model.add(Conv2D(32, (3, 3), activation='sigmoid', name='convo3'))
    self.Model.add(MaxPooling2D(pool_size=(2, 2)))
    self.Model.add(Conv2D(64, (3, 3), activation='relu', name='convo4'))
    self.Model.add(MaxPooling2D(pool_size=(2, 2)))
    self.Model.add(Flatten())

    self.Model.add(Dense(512))
    self.Model.add(Dropout(0.5))
    self.Model.add(BatchNormalization(scale=False))
    self.Model.add(Activation('relu'))
    self.Model.add(Dropout(0.5))
    self.Model.add(Dense(4, activation='softmax'))

额外验证点 #

• 确认steps_per_epoch计算正确：142657 // 16 = 8916，与代码中设置一致，无需修改。
• 数据生成器的target_size=(224,224)、color_mode='grayscale'与模型输入形状(224,224,1)匹配，无需调整。

内容的提问来源于stack exchange，提问作者Dave Kennedy