迁移学习训练模型时遇Dense层维度不匹配ValueError问题求助

错误信息 #

ValueError: Exception encountered when calling layer "dense" (type Dense).
Dimensions must be equal, but are 100352 and 2048 for '{{node dense/MatMul}} = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false](Placeholder, dense/MatMul/ReadVariableOp)' with input shapes: [?,100352], [2048,256].
Call arguments received by layer "dense" (type Dense):
inputs=tf.Tensor(shape=(None, 100352), dtype=float32)

迁移学习代码 #

import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
import matplotlib as plt
import tensorflow as tf
from tensorflow import keras
from keras import layers
from sklearn.model_selection import train_test_split
import pickle

#loading data
IMG_SIZE = 225
pickle_in = open("X.pickle", "rb")
X = pickle.load(pickle_in)
pickle_in = open("Y.pickle", "rb")
Y = pickle.load(pickle_in)
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.20, random_state=42)
X_train, X_val, Y_train, Y_val = train_test_split(X_train, Y_train, test_size=0.20, random_state=42)

#normalize
X_train /= 255.
X_test /= 255.

#loading VGG16
VGG16 = keras.models.load_model("VGG16.h5", compile=False, custom_objects = {"sin": tf.math.sin})

#model
model = keras.Sequential()
model.add(keras.Input(shape=(IMG_SIZE, IMG_SIZE, 3)))
for idx, layer in enumerate(VGG16.layers[0:-1]):
    model.add(layer)
    layer._name = "VGG16" + str(idx)
for layer in model.layers[1:]:
    layer.trainable = False
model.add(layers.Flatten())
model.add(layers.Conv2D(64, kernel_size=(3,3), padding="SAME", activation=tf.math.sin))
model.add(layers.BatchNormalization())
model.add(layers.MaxPooling2D(pool_size=(2,2), padding="SAME"), strides=2)
model.add(layers.Conv2D(32, kernel_size=(3,3), padding="SAME", activation=tf.math.sin))
model.add(layers.BatchNormalization())
model.add(layers.MaxPooling2D(pool_size=(2,2), padding="SAME"), strides=2)
model.add(layers.Flatten())
model.add(layers.Dense(1, activation="sigmoid"))
print(model.summary())

#training model
loss = keras.losses.BinaryCrossentropy()
optim = keras.optimizers.Adam(learning_rate=0.001)
model.compile(loss=loss, optimizer=optim, metrics=["accuracy"])
model.fit(X_train, Y_train, epochs=15, validation_data=(X_val, Y_val))
model.evaluate(X_test, Y_test, verbose=2)

原模型代码 #

import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
import tensorflow as tf
from tensorflow import keras
from tensorflow import math
from keras import layers
from keras.datasets import cifar10

def main():
    #loading data
    (X_train, Y_train), (X_test, Y_test) = keras.datasets.cifar10.load_data()
    X_test, X_train = X_test.astype("float32") / 255., X_train.astype("float32") / 255.
    Y_train, Y_test = keras.utils.to_categorical(Y_train, 10), keras.utils.to_categorical(Y_test, 10)

    #VGG16 model with SIREN
    model = keras.Sequential()
    model.add(layers.Conv2D(64, (3, 3), padding='same', kernel_initializer="he_uniform", activation=tf.math.sin, input_shape=(32,32,1)))
    model.add(layers.Conv2D(64, (3, 3), padding='same', activation=math.sin))
    model.add(layers.BatchNormalization())
    model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=2))
    model.add(layers.Conv2D(128, (3, 3), padding='same', activation=math.sin))
    model.add(layers.Conv2D(128, (3, 3), padding='same', activation=math.sin))
    model.add(layers.BatchNormalization())
    model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=2))
    model.add(layers.Conv2D(256, (3, 3), padding='same', activation=math.sin))
    model.add(layers.Conv2D(256, (3, 3), padding='same', activation=math.sin))
    model.add(layers.Conv2D(256, (3, 3), padding='same', activation=math.sin))
    model.add(layers.BatchNormalization())
    model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=2))
    model.add(layers.Conv2D(512, (3, 3), padding='same', activation=math.sin))
    model.add(layers.Conv2D(512, (3, 3), padding='same', activation=math.sin))
    model.add(layers.Conv2D(512, (3, 3), padding='same', activation=math.sin))
    model.add(layers.BatchNormalization())
    model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=2))
    model.add(layers.Flatten())
    model.add(layers.Dense(256, activation=math.sin))
    model.add(layers.Dense(128, activation=math.sin))
    model.add(layers.Dense(10, activation="softmax"))

    #training model
    lr = 0.0001
    loss = keras.losses.CategoricalCrossentropy()
    decayed_lr = tf.keras.optimizers.schedules.ExponentialDecay(lr, 10000, 0.85, True)
    optim = keras.optimizers.Adam(decayed_lr)
    model.compile(loss=loss, optimizer=optim, metrics=["accuracy"])
    checkpoint_filepath = '/Users/JamesRONewton/Documents/Programming/MachineLearning/SIREN_projects/BrainTumor/checkpoint.hdf5'
    checkpoint = keras.callbacks.ModelCheckpoint(filepath = checkpoint_filepath, monitor='accuracy', verbose=2, save_best_only=True, save_weights_only=True, mode='auto', save_freq ="epoch")
    try:
        model.load_weights(checkpoint_filepath)
    except Exception as e:
        pass
    model.fit(X_train, Y_train, batch_size=128, epochs=25, callbacks = [checkpoint])
    model.evaluate(X_test, Y_test, verbose=2)

    #saving model
    model.save("VGG16.h5")

if __name__ == '__main__':
    main()

用户疑问：使用了不同尺寸的输入，查阅资料得知只要步长一致就不会有问题，认为步长是一致的，请教如何解决维度不匹配问题。

问题分析与解决方案 #

1. 核心问题：错误引入原模型的全连接层 #

原模型的结构最后包含Flatten+Dense(256)+Dense(128)+Dense(10)，你在迁移时使用VGG16.layers[0:-1]，仅排除了最后一层Dense(10)，把前面的Dense(256)、Dense(128)都加入了新模型。这些全连接层的权重是基于原模型输入尺寸（32x32x1）计算的，其输入维度固定为2048（原模型Flatten后的特征数），但新输入（225x225x3）经过卷积池化后Flatten的特征数为100352，导致维度不匹配，触发错误。

2. 次要问题：输入通道不匹配 #

原模型第一层卷积的输入通道是1（对应input_shape=(32,32,1)），但新模型输入是3通道（(225,225,3)），这会导致第一层卷积的输入维度不兼容。

3. 结构语法错误 #

• 你在Flatten后添加Conv2D层，而Conv2D需要4D张量（(batch, height, width, channels)），但Flatten输出是2D张量（(batch, features)），这会导致后续层维度混乱。
• MaxPooling2D的语法错误：strides=2应该放在层的构造函数括号内，而非作为单独参数传入model.add()。

修正后的迁移学习代码 #

import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
import tensorflow as tf
from tensorflow import keras
from keras import layers
from sklearn.model_selection import train_test_split
import pickle

# Loading data
IMG_SIZE = 225
pickle_in = open("X.pickle", "rb")
X = pickle.load(pickle_in)
pickle_in = open("Y.pickle", "rb")
Y = pickle.load(pickle_in)
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.20, random_state=42)
X_train, X_val, Y_train, Y_val = train_test_split(X_train, Y_train, test_size=0.20, random_state=42)

# Normalize
X_train /= 255.
X_test /= 255.
X_val /= 255.  # 补充验证集归一化

# Load pre-trained model
VGG16 = keras.models.load_model("VGG16.h5", compile=False, custom_objects={"sin": tf.math.sin})

# 仅提取原模型的卷积基（去掉所有全连接层：Flatten + Dense层）
conv_base = keras.Sequential()
# 处理输入通道不匹配：替换第一层卷积为3输入通道
conv_base.add(layers.Conv2D(64, (3, 3), padding='same', kernel_initializer="he_uniform", activation=tf.math.sin, input_shape=(IMG_SIZE, IMG_SIZE, 3)))
# 添加原模型剩余的卷积池化层（从第2层到倒数第4层，因为倒数4层是Flatten、Dense256、Dense128、Dense10）
for idx, layer in enumerate(VGG16.layers[1:-4]):
    conv_base.add(layer)
    layer._name = "VGG16_" + str(idx+1)

# 冻结卷积基
for layer in conv_base.layers:
    layer.trainable = False

# 构建新模型：卷积基 + 自定义分类头
model = keras.Sequential([
    conv_base,
    # 在卷积基后添加自定义卷积层（此时输入是4D张量）
    layers.Conv2D(64, kernel_size=(3,3), padding="SAME", activation=tf.math.sin),
    layers.BatchNormalization(),
    layers.MaxPooling2D(pool_size=(2,2), padding="SAME", strides=2),
    layers.Conv2D(32, kernel_size=(3,3), padding="SAME", activation=tf.math.sin),
    layers.BatchNormalization(),
    layers.MaxPooling2D(pool_size=(2,2), padding="SAME", strides=2),
    # 最后Flatten后接分类层
    layers.Flatten(),
    layers.Dense(1, activation="sigmoid")
])

print(model.summary())

# Training
loss = keras.losses.BinaryCrossentropy()
optim = keras.optimizers.Adam(learning_rate=0.001)
model.compile(loss=loss, optimizer=optim, metrics=["accuracy"])
model.fit(X_train, Y_train, epochs=15, validation_data=(X_val, Y_val))
model.evaluate(X_test, Y_test, verbose=2)

关键修正点说明 #

• 仅保留卷积基：原模型的全连接层依赖固定输入维度，迁移学习时应只保留卷积特征提取部分，避免引入维度不兼容的层。
• 解决通道不匹配：通过替换第一层卷积为3输入通道，适配新数据的RGB格式；也可以选择在输入后添加layers.Conv2D(1, (1,1), activation='sin')将3通道转为1通道，保留原第一层卷积权重。
• 修正层结构顺序：确保Conv2D层在4D张量输入后添加，Flatten仅在最后连接全分类层前使用。
• 修正语法错误：将strides=2移至MaxPooling2D的构造函数参数内。

内容的提问来源于stack exchange，提问作者James Newton