运行科研论文代码时遇模型输入输出形状不匹配错误求助

尝试运行科研论文代码执行训练脚本时，遇到模型输入输出形状不兼容问题，报错提到zero_padding2d层，但代码中并未显式使用该层。

代码片段 #

network = networks[str(config[parser_args.task]['NETWORK'])]

for idx, row in grid_df.iterrows(): 
    if row['timestamp'] == 'None':
        learning_rate = row['lr']
        spatial_dropout_rate = row['s_d']
        temporal_dropout_rate = row['t_d']
        concat_dropout_rate = row['c_d']
        curr_time_n = datetime.datetime.now()
        curr_time = curr_time_n.strftime("%Y-%m-%d %H:%M:%S")
        time_passed_since_start = curr_time_n - start_time_n
        time_passed_since_start_in_min = time_passed_since_start/datetime.timedelta(minutes=1)
        time_passed_since_start_in_hours = time_passed_since_start_in_min/60

        if time_passed_since_start_in_hours >= 50.:
            grid_df.to_csv(grid_file, index=False)
            sys.exit(0)
        else:
            grid_df.loc[idx, 'timestamp'] = curr_time

            train_generator = crop_generator(input_path=tr_path, batch_size=batch_size, mode="train", do_shuffle=True, epsilon=0)
            val_generator = crop_generator(input_path=val_path, batch_size=batch_size, mode="val", do_shuffle=True, epsilon=0)

            # 空间分支（CNN）
            cnn_base_model = network(include_top=False, weights = 'imagenet',classes=num_classes)
            cnn_output = cnn_base_model.output
            img_avgpool = GlobalAveragePooling2D()(cnn_output)
            img_dense_1 = Dense(512, activation="relu")(img_avgpool)
            img_dropout = Dropout(spatial_dropout_rate)(img_dense_1)
            img_dense_2 = Dense(256, activation="relu")(img_dropout)

            # 时序分支（LSTM）
            temporal_input_layer = Input(batch_shape = (None, 23, 1), name='time_input_layer')
            lstm_1 = LSTM(100, input_shape=(23,1), dropout=temporal_dropout_rate)(temporal_input_layer)
            ts_output = Dense(32, activation="relu")(lstm_1)

            # 融合分支
            final_merged = concatenate([img_dense_2, ts_output])
            concat_dropout = Dropout(concat_dropout_rate)(final_merged)
            final_dense = Dense(32, activation="relu")(concat_dropout)
            predictions = Dense(num_classes, activation='softmax')(final_dense)

            optimizer = Adam(lr=0.0005)
            model = Model(inputs=[cnn_base_model.input, temporal_input_layer], outputs=predictions)
            model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) 

            print(model.summary())
            # 后续的model.fit调用报错

完整报错信息 #

File "train-gridsearch.py", line 203, in <module>
    history = model.fit(train_generator,  
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1187, in fit
    tmp_logs = self.train_function(iterator)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
File "/tmp/__autograph_generated_file52u8rjgw.py", line 15, in tf__train_function
    retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
File "/usr/local/lib/python3.8/dist-packages/keras/engine/input_spec.py", line 232, in assert_input_compatibility
    raise ValueError(
ValueError: in user code:

    File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 70, in error_handler
        raise e.with_traceback(filtered_tb) from None
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/input_spec.py", line 232, in assert_input_compatibility
        raise ValueError(

    ValueError: Input 0 of layer "zero_padding2d" is incompatible with the layer: expected ndim=4, found ndim=2. Full shape received: (None, 1)

解决方案 #

1. 明确问题根源

zero_padding2d是你使用的预训练CNN模型（如ResNet、VGG等）内部的层，这类卷积层要求输入为4维张量：(batch_size, 图像高度, 图像宽度, 通道数)，但你的生成器返回的图像数据是2维的(None,1)，完全不符合要求。

2. 检查并修复生成器输出

• 在crop_generator函数中添加打印语句，确认返回数据的形状：

  def crop_generator(input_path, batch_size, mode, do_shuffle, epsilon):
      # 原有生成逻辑
      # 假设生成器返回 [图像数据, 时序数据], 标签
      x_image, x_temporal, y_label = ... 
      print(f"图像形状: {x_image.shape}, 时序数据形状: {x_temporal.shape}")
      yield [x_image, x_temporal], y_label
  

• 确保图像数据形状与预训练CNN的输入要求一致：比如用ResNet50的话，默认输入是(batch_size, 224, 224, 3)（RGB图像），单通道图像则为(batch_size, h, w, 1)。
• 时序数据需匹配(batch_size, 23, 1)的形状，与你定义的temporal_input_layer一致。

3. 验证模型输入兼容性

手动构造符合形状的测试数据，验证模型前向传播是否正常：

import numpy as np

# 假设CNN输入为224x224x3
test_img = np.random.rand(batch_size, 224, 224, 3)
# 时序数据匹配(23,1)的维度
test_ts = np.random.rand(batch_size, 23, 1)

# 测试模型预测
preds = model.predict([test_img, test_ts])
print(f"预测输出形状: {preds.shape}") # 应输出 (batch_size, num_classes)

如果该测试能正常运行，说明模型结构无问题，问题完全出在生成器的输出形状上。

4. 对齐预训练模型输入尺寸

如果预训练CNN的输入尺寸与你的图像数据不符，可在加载模型时指定input_shape参数：

# 示例：指定输入为256x256x3
cnn_base_model = network(include_top=False, weights='imagenet', 
                         input_shape=(256, 256, 3), classes=num_classes)

同时确保生成器输出的图像尺寸与该参数一致。

内容的提问来源于stack exchange，提问作者Muhammad Omer Mansoor