使用TensorFlow构建VGG19模型时出现形状不兼容错误求助

错误信息 #

Epoch 1/10
2023-11-07 10:50:42.834617: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 192675840 exceeds 10% of free system memory.
2023-11-07 10:50:43.002592: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 192675840 exceeds 10% of free system memory.
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
/workspaces/SI-GuidedProject-592631-1697551698/Project Development Phase/Model Build.ipynb Cell 11 line 1
----> 1 vgm.fit(x_train,epochs=10,validation_data=x_test)

File ~/.python/current/lib/python3.10/site-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     65 except Exception as e:  # pylint: disable=broad-except
     66   filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67   raise e.with_traceback(filtered_tb) from None
     68 finally:
     69   del filtered_tb

File ~/.python/current/lib/python3.10/site-packages/tensorflow/python/eager/execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     52 try:
     53   ctx.ensure_initialized()
---> 54   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     55                                       inputs, attrs, num_outputs)
     56 except core._NotOkStatusException as e:
     57   if name is not None:

InvalidArgumentError: Graph execution error:

Detected at node 'Equal' defined at (most recent call last):
    File "/home/codespace/.python/current/lib/python3.10/runpy.py", line 196, in _run_module_as_main
      return _run_code(code, main_globals, None,
...
    File "/home/codespace/.python/current/lib/python3.10/site-packages/keras/utils/metrics_utils.py", line 893, in sparse_categorical_matches
      matches = tf.cast(tf.equal(y_true, y_pred), backend.floatx())
Node: 'Equal'
Incompatible shapes: [15,7,7] vs. [15]
     [[{{node Equal}}]] [Op:__inference_train_function_2441]

问题分析 #

错误提示的Incompatible shapes: [15,7,7] vs. [15]说明模型输出形状和训练数据的标签形状不匹配。

查看代码可知：你仅在VGG19的输出后定义了Flatten和Dense层，但没有将这些层与原VGG19模型拼接成完整的新模型。vgm仍然是原始的VGG19模型（include_top=False），其输出形状为(None,7,7,512)，自定义的顶层并未被整合进vgm，导致训练时模型输出的是7×7的特征图，而数据生成器输出的是形状为(15,63)的独热编码标签（class_mode='categorical'），两者形状无法匹配，引发错误。

同时模型摘要显示Trainable params: 0，也证明你添加的Dense层并未被包含在训练模型中。

解决步骤 #

修改代码，使用keras.Model构建包含自定义顶层的完整模型：

修正后的完整代码 #

from keras.preprocessing.image import ImageDataGenerator
from keras.applications.vgg19 import VGG19
from keras.layers import Dense, Flatten
from keras.models import Model

train_datagen = ImageDataGenerator(rescale =1./255)

x_train = train_datagen.flow_from_directory(
    'train data path',
    target_size=(224,224),
    class_mode = 'categorical',
    batch_size = 15
)

# 加载预训练VGG19，不包含顶层
base_model = VGG19(input_shape=(224, 224, 3), weights='imagenet', include_top=False)
# 冻结预训练层
for layer in base_model.layers:
    layer.trainable = False

# 添加自定义顶层
x = Flatten()(base_model.output)
output_layer = Dense(63, activation='softmax')(x)

# 构建完整模型
vgm = Model(inputs=base_model.input, outputs=output_layer)

# 查看模型摘要，确认顶层已添加
vgm.summary()

# 编译模型
vgm.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# 开始训练
vgm.fit(x_train, epochs=1)

关键修改点 #

• 导入keras.models.Model类，用于构建自定义完整模型
• 将原vgm重命名为base_model作为基础特征提取器
• 使用Model(inputs=..., outputs=...)将基础模型与自定义顶层拼接成完整可训练模型
• 此时vgm的输出形状为(None,63)，与数据生成器的标签形状完全匹配

额外说明 #

对于内存警告Allocation of 192675840 exceeds 10% of free system memory，可尝试以下优化：

• 减小batch_size（比如从15调整为8或4）
• 若有GPU资源，切换到GPU运行TensorFlow，可大幅降低CPU内存占用压力

内容的提问来源于stack exchange，提问作者Rueben V Philip