自定义DataGenerator训练InceptionV4时形状不兼容问题解决咨询

使用8通道npy格式数据，自定义Keras的CustomDataGenerator读取数据，训练InceptionV4模型时触发错误：

ValueError: Shapes (None, None, None, None) and (None, 6) are incompatible

已知InceptionV4输入形状为(batch_size, height, width, channel)，自定义生成器输出符合该形状，标签已转one-hot格式，且该生成器此前用于Unet训练无问题。相关代码及模型结构如下：

自定义生成器代码 #

params = {'dim': (299,299),
          'batch_size': 1,
          'n_classes': 6,
          'n_channels': 8,}
class CustomDataGenerator(Sequence):
    def __init__(self, image_folders, label_folders, dir, dim=(299,299),  batch_size=1,n_classes=6,n_channels=8,shuffle=True):
        self.image_folders = image_folders
        ...
        self.on_epoch_end()

    def __len__(self):
        return int(np.ceil(len(self.image_paths) / self.batch_size))  

    def __getitem__(self, index):
        batch_image_paths = self.image_paths[index * self.batch_size: (index + 1) * self.batch_size]
        batch_label_paths = self.label_paths[index * self.batch_size: (index + 1) * self.batch_size]
        batch = zip(batch_image_paths, batch_label_paths)

        return self.get_data(batch)

    def on_epoch_end(self):
        self.image_paths = []
        self.label_paths = []
        for folder in self.image_folders:
            image_folder_path = os.path.join(self.dir, folder)
            image_files = os.listdir(image_folder_path)
            for file_name in image_files:
                self.image_paths.append(os.path.join(image_folder_path, file_name))

        for folder in self.label_folders:
            ...

        if self.shuffle:
            ...

    def get_data(self, batch):
        X = np.empty((self.batch_size, *self.dim, self.n_channels))
        y = np.empty((self.batch_size, *self.dim, self.n_classes))

        for i, (image_path, label_path) in enumerate(batch):
            image = np.load(image_path)
            label = np.load(label_path)
            self.y_true.append(label)

            label_grayscale = np.mean(label, axis=-1)
            min_val = label_grayscale.min()
            max_val = label_grayscale.max()
            scaled_label = (label_grayscale - min_val) / (max_val - min_val)  # Scale to [0, 1]
            scaled_label = (scaled_label * (self.n_classes - 1)).astype(int)  # Scale to [0, n_classes-1]
            label_one_hot = to_categorical(scaled_label, num_classes=self.n_classes)

            X[i,] = image
            y[i,] = label_one_hot

        return X, y
train_datagen = CustomDataGenerator(image_folders, label_folders, train_dir, **params, shuffle = True)

model = model_incetionV4
model.compile(optimizer=Adam(lr=0.0001), loss=categorical_crossentropy, metrics=['accuracy'])
model_checkpoint = ModelCheckpoint('weight/123467_InceptionV4_test1.hdf5', monitor='loss',verbose=1, save_best_only=True)
model.fit(train_datagen, steps_per_epoch=30, epochs=500, validation_data=val_datagen, callbacks=[model_checkpoint])

InceptionV4模型代码 #

class InceptionV4(tf.keras.Model):
    def __init__(self,blocksList,num_classes):
        ...
    def call(self,inputs,training=None):
        x=self.stem(inputs)

        x=self.inceptionA(x)
        x=self.redA(x)
        x=self.inceptionB(x)
        x=self.redB(x)
        x=self.inceptionC(x)

        x=self.avgpool(x)
        x=self.dropout(x)
        x=self.dense(x)
        x=self.softmax(x)

model_incetionV4=InceptionV4(blocksList=[4,7,3],num_classes=6)
batch_size = 1
model_incetionV4.build(input_shape=(batch_size,299,299,8))
model_incetionV4.summary()

模型结构摘要 #

Model: "inception_v4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
stem (Sequential)            (1, 35, 35, 384)          609760
_________________________________________________________________
inceptionA (Sequential)      (1, 35, 35, 384)          1277824   
_________________________________________________________________
reducetionA (Sequential)     (1, 17, 17, 1024)         2309280
_________________________________________________________________
inceptionB (Sequential)      (1, 17, 17, 1024)         18135040  
_________________________________________________________________
reducetionB (Sequential)     (1, 8, 8, 1536)           2571456
_________________________________________________________________
inceptionC (Sequential)      (1, 8, 8, 1536)           31940160  
_________________________________________________________________
global_average_pooling2d (Gl multiple                  0
_________________________________________________________________
dropout (Dropout)            multiple                  0
_________________________________________________________________
dense (Dense)                multiple                  9222
_________________________________________________________________
activation_21 (Activation)   multiple                  0
=================================================================

错误核心是生成器输出的标签形状与模型输出形状不匹配：

• InceptionV4是图像分类模型，最终输出为(batch_size, num_classes)（即(1,6)），对应全局池化后接全连接层的单样本单类别输出。
• 自定义生成器中标签y被定义为(batch_size, *self.dim, self.n_classes)（即(1,299,299,6)），这是图像分割任务的像素级标签格式（适配Unet），完全不符合分类模型的输入要求。

需要修改自定义生成器，将标签从像素级one-hot调整为样本级one-hot：

修改生成器的关键代码 #

在get_data方法中，调整标签形状和处理逻辑：

def get_data(self, batch):
    # 输入X的定义保持不变
    X = np.empty((self.batch_size, *self.dim, self.n_channels))
    # 修改y的形状：从像素级转为样本级
    y = np.empty((self.batch_size, self.n_classes))

    for i, (image_path, label_path) in enumerate(batch):
        image = np.load(image_path)
        label = np.load(label_path)
        self.y_true.append(label)

        label_grayscale = np.mean(label, axis=-1)
        min_val = label_grayscale.min()
        max_val = label_grayscale.max()
        scaled_label = (label_grayscale - min_val) / (max_val - min_val)
        scaled_label = (scaled_label * (self.n_classes - 1)).astype(int)
        
        # 从像素级标签提取样本级标签：示例为取出现次数最多的类别，可根据任务需求调整
        sample_label = np.bincount(scaled_label.flatten()).argmax()
        # 转换为样本级one-hot编码
        label_one_hot = to_categorical(sample_label, num_classes=self.n_classes)

        X[i,] = image
        y[i,] = label_one_hot

    return X, y

模型部分无需修改 #

从模型结构摘要可知，InceptionV4经过全局平均池化后，将(1,8,8,1536)的特征转为(1,1536)，再通过全连接层输出(1,6)，完全符合分类任务的输出形状，无需调整。

内容的提问来源于stack exchange，提问作者Syuuuu