TensorFlow多输入CNN模型训练报错：期望2个输入却收到1个张量

问题描述 #

作为TensorFlow新手，尝试基于MobileNet和ResNet架构构建CNN模型，编译训练时触发如下错误：

ValueError: Layer "model" expects 2 input(s), but it received 1 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(None, None, None, None) dtype=float32>]

用户实现代码如下：

import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.applications import MobileNetV3Large, ResNet50V2
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Concatenate, Conv2D, Flatten, Dense, Dropout

train_path = "trainpath"
test_path = "testpath"
valid_path = "validpath"

datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    vertical_flip=True
)

train_data = datagen.flow_from_directory(
    train_path,
    target_size=(224,224),
    batch_size=32,
    class_mode="categorical",
    shuffle=True
)

valid_data = datagen.flow_from_directory(
    valid_path,
    target_size=(224,224),
    batch_size=32,
    class_mode="categorical",
    shuffle=True
)

test_data = datagen.flow_from_directory(
    test_path,
    target_size=(224,224),
    batch_size=32,
    class_mode="categorical",
    shuffle=True
)

mobile_net = MobileNetV3Large(include_top=False, weights='imagenet', input_shape=(224, 224, 3))
res_net = ResNet50V2(include_top=False, weights='imagenet', input_shape=(224, 224, 3))

mobile_net.trainable = False
res_net.trainable = False

mobile_net_output = mobile_net.output
res_net_output = res_net.output

concatenated = Concatenate()([mobile_net_output, res_net_output])

x = Conv2D(filters=256, kernel_size=(3, 3), activation='relu', padding='same')(concatenated)
x = Conv2D(filters=256, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Dropout(0.25)(x)  # Dropout layer to reduce overfitting
x = Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Dropout(0.25)(x)  # Dropout layer to reduce overfitting
x = Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Dropout(0.25)(x)  
x = Flatten()(x)
x = Dense(512, activation='relu')(x)
x = Dropout(0.5)(x) 
x = Dense(256, activation='relu')(x)
output = Dense(3, activation='softmax')(x)  

model = Model(inputs=[mobile_net.input, res_net.input], outputs=output)

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, validation_data = valid_data, epochs=15, batch_size=32, verbose=True)

问题根源 #

代码中定义的模型要求两个独立输入（MobileNet和ResNet各自的输入层），但ImageDataGenerator生成的训练数据每次只输出一组图像张量，导致输入数量不匹配，触发错误。

修正方案 #

让两个预训练模型共享同一个输入层，这样模型仅需要一个输入，与数据生成器的输出格式匹配。修改步骤如下：

1. 创建统一的输入层，替换原来分别给两个模型指定输入形状的方式
2. 将两个预训练模型绑定到这个统一输入层上
3. 重新定义模型的输入为这个统一层

完整修正后的代码 #

import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.applications import MobileNetV3Large, ResNet50V2
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Concatenate, Conv2D, Flatten, Dense, Dropout

train_path = "trainpath"
test_path = "testpath"
valid_path = "validpath"

datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    vertical_flip=True
)

train_data = datagen.flow_from_directory(
    train_path,
    target_size=(224,224),
    batch_size=32,
    class_mode="categorical",
    shuffle=True
)

valid_data = datagen.flow_from_directory(
    valid_path,
    target_size=(224,224),
    batch_size=32,
    class_mode="categorical",
    shuffle=True
)

test_data = datagen.flow_from_directory(
    test_path,
    target_size=(224,224),
    batch_size=32,
    class_mode="categorical",
    shuffle=True
)

# 共享输入层
input_layer = tf.keras.layers.Input(shape=(224, 224, 3))
mobile_net = MobileNetV3Large(include_top=False, weights='imagenet', input_tensor=input_layer)
res_net = ResNet50V2(include_top=False, weights='imagenet', input_tensor=input_layer)

mobile_net.trainable = False
res_net.trainable = False

mobile_net_output = mobile_net.output
res_net_output = res_net.output

concatenated = Concatenate()([mobile_net_output, res_net_output])

x = Conv2D(filters=256, kernel_size=(3, 3), activation='relu', padding='same')(concatenated)
x = Conv2D(filters=256, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Dropout(0.25)(x)  # Dropout layer to reduce overfitting
x = Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Dropout(0.25)(x)  # Dropout layer to reduce overfitting
x = Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same')(x)
x = Dropout(0.25)(x)  
x = Flatten()(x)
x = Dense(512, activation='relu')(x)
x = Dropout(0.5)(x) 
x = Dense(256, activation='relu')(x)
output = Dense(3, activation='softmax')(x)  

# 模型输入为统一的input_layer
model = Model(inputs=input_layer, outputs=output)

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, validation_data = valid_data, epochs=15, batch_size=32, verbose=True)

原理说明 #

通过共享输入层，同一组图像数据会同时传入MobileNet和ResNet进行特征提取，模型最终只需要接收一个输入张量，和ImageDataGenerator生成的数据格式完全匹配，从而解决输入数量不匹配的问题。

内容的提问来源于stack exchange，提问作者amitaf