Keras多输入单标签自定义数据集生成器训练报错求助

AI 小北

2026-7-9

多输入Keras模型与自定义Sequence数据生成器不匹配问题

我编写了Python脚本加载无法直接入内存的大型数据集，喂给Keras模型。自定义数据集代码如下：

import numpy as np
import tensorflow as tf

class CustomDataSequence(tf.keras.utils.Sequence):
    def __init__(self, image_set, gender,Slice,Type, label_set, batch_size=32, image_size=(256, 256)):
        self.image_set = np.array(image_set)
        self.Gender = gender
        self.Slice = Slice
        self.Type = Type
        self.label_set = label_set
        self.batch_size = batch_size
        self.image_size = image_size
    def __get_image(self, image):
        image = tf.keras.preprocessing.image.load_img(image, color_mode='rgb', target_size=self.image_size)
        image_arr = tf.keras.preprocessing.image.img_to_array(image)
        return image_arr/255

    def __get_data(self, images, gender,Slice,Type, labels):
        image_batch = np.asarray([self.__get_image(img) for img in images])
        label_batch = labels
        Gender_batch = gender
        Slice_batch = Slice
        Type_batch = Type
        return [image_batch, Gender_batch, Slice_batch, Type_batch], label_batch

    def __getitem__(self, index):
        images = self.image_set[index * self.batch_size:(index + 1) * self.batch_size]
        Gender = self.Gender[index * self.batch_size:(index + 1) * self.batch_size]

        Slice = self.Slice[index * self.batch_size:(index + 1) * self.batch_size]
        Type = self.Type[index * self.batch_size:(index + 1) * self.batch_size]

        labels = self.label_set[index * self.batch_size:(index + 1) * self.batch_size]
        data, labels = self.__get_data(images, Gender,Slice,Type, labels)
        return data, labels

    def __len__(self):
        return len(self.image_set) // self.batch_size + (len(self.image_set) % self.batch_size > 0)

该生成器包含多个输入变量（image_batch、Gender_batch、Slice_batch、Type_batch）。主代码中创建训练和验证数据集对象的代码：

train_ds = CustomDataSequence(Train,Train_G,Train_S,Train_T, Train_y, image_size=(80,80), batch_size=32)
val_ds = CustomDataSequence(Validation,Val_G,Val_S,Val_T, Val_y, image_size=(80,80), batch_size=32)

模型结构代码：

model = ResNet50(input_shape = IMAGE_SIZE,include_top = False, weights = '/content/RadImageNet-ResNet50_notop.h5')
for layer in model.layers[0:len(model.layers)-30]:
     layer.trainable = False
inputB = Input(shape=(1,))
inputC = Input(shape=(Val_S.shape[1],))
inputD = Input(shape=(Val_T.shape[1],))
model_output = GlobalMaxPooling2D()(model.output)
model_output = concatenate([model_output, inputB,inputC,inputD])
model_output = Dense(64, activation='relu')(model_output)
model_output = Dense(32, activation='relu')(model_output)
model_output = Dense(16, activation='relu')(model_output)
model_output = Dense(1, activation='linear')(model_output)
model=Model(inputs=[model.input,inputB,inputC,inputD],outputs=model_output)

执行model.fit时出现错误：

InvalidArgumentError: Graph execution error:
TypeError: `generator` yielded an element that did not match the expected structure. The expected structure was ((TensorSpec(shape=(None, None, None, None), dtype=tf.float32, name=None), TensorSpec(shape=(None,), dtype=tf.int64, name=None), SparseTensorSpec(TensorShape([None, None]), tf.float32), SparseTensorSpec(TensorShape([None, None]), tf.float32)), TensorSpec(shape=(None,), dtype=tf.int64, name=None)), but the yielded element was ((array([[[[0.4392157 , 0.4392157 , 0.4392157 ],
         [0.4862745 , 0.4862745 , 0.4862745 ],
         [0.52156866, 0.52156866, 0.52156866],
         ...,
         [0.8784314 , 0.8784314 , 0.8784314 ],
         [0.8901961 , 0.8901961 , 0.8901961 ],
         [0.9019608 , 0.9019608 , 0.9019608 ]],

        [[0.49019608, 0.49019608, 0.49019608],
         [0.53333336, 0.53333336, 0.53333336],
         [0.56078434, 0.56078434, 0.56078434],
         ...,

仅输入图像时代码运行正常，添加多输入后出现此问题，尝试用字典传递输入也未解决。

问题原因

错误提示显示生成器返回的数据结构与模型期望不匹配：模型期望部分输入为SparseTensorSpec，但生成器返回的是普通numpy数组；同时部分输入的形状未对齐（比如Gender输入的维度）。

修复方案

1. 修正输入数据的形状匹配模型定义

模型中inputB的shape为(1,)，因此Gender_batch需要是(batch_size, 1)的二维数组，而非(batch_size,)的一维数组。
确保Slice_batch和Type_batch的形状与模型中inputC、inputD定义的(Val_S.shape[1],)、(Val_T.shape[1],)完全匹配，即保持(batch_size, feature_num)的二维结构。

2. 修改CustomDataSequence的__get_data方法

调整返回的输入数组形状，确保与模型输入一致：

def __get_data(self, images, gender, Slice, Type, labels):
    image_batch = np.asarray([self.__get_image(img) for img in images])
    # 给Gender增加一个维度，匹配inputB的(1,)输入形状
    Gender_batch = np.expand_dims(gender, axis=1)
    # 确保Slice和Type是二维数组，避免被识别为稀疏张量
    Slice_batch = np.array(Slice)
    Type_batch = np.array(Type)
    label_batch = np.array(labels)
    return [image_batch, Gender_batch, Slice_batch, Type_batch], label_batch

3. 验证数据类型一致性

确保所有输入的数据类型与模型期望一致：

图像数据已归一化到0-1，为float32，无需调整。
若模型期望其他输入为float32，可转换数组类型，例如：
```
Gender_batch = np.expand_dims(gender, axis=1).astype(np.float32)
```

4. 可选：用字典传递输入（更清晰）

给模型的每个Input层指定名称，然后生成器返回字典形式的输入：

模型定义时添加输入名称：

inputB = Input(shape=(1,), name="gender_input")
inputC = Input(shape=(Val_S.shape[1],), name="slice_input")
inputD = Input(shape=(Val_T.shape[1],), name="type_input")
# 模型输入使用字典映射
model=Model(
    inputs={"input_1": model.input, "gender_input": inputB, "slice_input": inputC, "type_input": inputD},
    outputs=model_output
)

生成器的__get_data方法返回字典：

return {
    "input_1": image_batch,
    "gender_input": Gender_batch,
    "slice_input": Slice_batch,
    "type_input": Type_batch
}, label_batch

内容的提问来源于stack exchange，提问作者Imaney