使用Keras+VGG16处理DICOM图像时遇ValueError问题求助

我尝试用Keras和VGG16基于DICOM图像构建深度学习模型，自定义了数据生成器处理图像，但调用fit()时出现ValueError: Input arrays must be multi-channel 2D images.错误。

自定义DICOM数据生成器代码 #

# tested on tf 2.1 
from keras_preprocessing.image.dataframe_iterator import DataFrameIterator
import numpy as np
import tensorflow as tf
import tensorflow_io as tfio

class DCMDataFrameIterator(DataFrameIterator):
    def __init__(self, *arg, **kwargs):
        self.white_list_formats = ('dcm')
        super(DCMDataFrameIterator, self).__init__(*arg, **kwargs)
        self.dataframe = kwargs['dataframe']
        self.x = self.dataframe[kwargs['x_col']]
        self.y = self.dataframe[kwargs['y_col']]
        self.color_mode = kwargs['color_mode']
        self.target_size = kwargs['target_size']

    def _get_batches_of_transformed_samples(self, indices_array):
        # get batch of images
        batch_x = np.array([self.read_dcm_as_array(dcm_path, self.target_size, color_mode=self.color_mode)
                            for dcm_path in self.x.iloc[indices_array]])
        
        batch_y = np.array(self.y.iloc[indices_array].astype(np.uint8))  # astype because y was passed as str

        # transform images
        if self.image_data_generator is not None:
            for i, (x, y) in enumerate(zip(batch_x, batch_y)):
                transform_params = self.image_data_generator.get_random_transform(x.shape)
                batch_x[i] = self.image_data_generator.apply_transform(x, transform_params)
                # you can change y here as well, eg: in semantic segmentation you want to transform masks as well 
                # using the same image_data_generator transformations.

        return batch_x, batch_y

    @staticmethod
    def read_dcm_as_array(dcm_path, target_size=(256, 256), color_mode='rgb'):
        img = tf.io.read_file(dcm_path)
        img = tfio.image.decode_dicom_image(img, dtype=tf.uint16)  
        img = tf.image.resize(img, target_size)
        img = tf.image.grayscale_to_rgb(img, name=None) # convert image grayscale to rgb for model VG16
        #img = np.expand_dims(img, -1)
        return img

数据增强与训练参数配置 #

# you can use preprocessing_function instead of rescale in all generators
# if you are using a pretrained network
train_augmentation_parameters = dict(
rescale=1.0/255.0,
rotation_range=10,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest',
brightness_range = [0.8, 1.2],
validation_split = 0.2
)

valid_augmentation_parameters = dict(
rescale=1.0/255.0,
validation_split = 0.2
)

test_augmentation_parameters = dict(
rescale=1.0/255.0
)

# training parameters
BATCH_SIZE = 32
CLASS_MODE = 'sparse'
COLOR_MODE = 'grayscale'
TARGET_SIZE = (300, 300)
EPOCHS = 10
SEED = 1337

train_consts = {
'seed': SEED,
'batch_size': BATCH_SIZE,
'class_mode': CLASS_MODE,
'color_mode': COLOR_MODE,
'target_size': TARGET_SIZE,  
'subset': 'training'
}

valid_consts = {
'seed': SEED,
'batch_size': BATCH_SIZE,
'class_mode': CLASS_MODE,
'color_mode': COLOR_MODE,
'target_size': TARGET_SIZE, 
'subset': 'validation'
}

test_consts = {
'batch_size': 1,  # should be 1 in testing
'class_mode': CLASS_MODE,
'color_mode': COLOR_MODE,
'target_size': TARGET_SIZE,  # resize input images
'shuffle': False
}

生成器初始化结果 #

Found 7828 validated image filenames belonging to 4 classes.
Found 1956 validated image filenames belonging to 4 classes.

模型构建代码 #

from keras.applications.vgg16 import VGG16
from keras.models import Sequential
from keras.layers import GlobalAveragePooling2D, Dense, Dropout

base_model = VGG16(weights='imagenet', include_top=False) 

n_class = 4 # 

# Freezer les couches du VGG16
for layer in base_model.layers: 
    layer.trainable = False

model = Sequential()
model.add(base_model) # Ajout du modèle VGG16
model.add(GlobalAveragePooling2D()) 
model.add(Dense(1024,activation='relu'))
model.add(Dropout(rate=0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(rate=0.2))
model.add(Dense(n_class, activation='softmax'))

错误信息 #

ValueError: Input arrays must be multi-channel 2D images.

解决方案 #

错误根源 #

1. 方法缩进错误：自定义生成器中的_get_batches_of_transformed_samples和read_dcm_as_array未正确缩进为类成员方法，导致父类方法未被重载，数据生成逻辑失效。
2. 通道配置冲突：VGG16预训练模型要求输入为3通道RGB图像，但参数中COLOR_MODE设为grayscale，且生成器返回的Tensor未正确转为numpy数组，通道维度不匹配。
3. 数据类型问题：read_dcm_as_array返回的是Tensor对象，直接转为numpy数组时可能出现维度异常。

修复步骤 #

1. 修正生成器方法缩进

确保_get_batches_of_transformed_samples和read_dcm_as_array缩进为DCMDataFrameIterator类的成员方法（参考上面修正后的生成器代码）。

2. 统一通道配置

将所有参数中的COLOR_MODE改为rgb，与VGG16的输入要求一致：

COLOR_MODE = 'rgb'

3. 修正图像读取逻辑

修改read_dcm_as_array方法，确保输出为3通道numpy数组：

@staticmethod
def read_dcm_as_array(dcm_path, target_size=(256, 256), color_mode='rgb'):
    img = tf.io.read_file(dcm_path)
    img = tfio.image.decode_dicom_image(img, dtype=tf.uint16)  
    # 确保图像为单通道格式后再转RGB
    if len(img.shape) == 2:
        img = tf.expand_dims(img, -1)
    img = tf.image.resize(img, target_size)
    if color_mode == 'rgb':
        img = tf.image.grayscale_to_rgb(img)
    # 转为numpy数组并调整数据类型
    return img.numpy().astype(np.float32)

4. 验证输入维度

在_get_batches_of_transformed_samples方法中添加调试代码，确认输出的batch维度正确：

print(f"Batch shape: {batch_x.shape}")  # 预期输出类似 (32, 300, 300, 3)

完成以上修改后，重新运行模型训练，即可解决输入格式不匹配的问题。

内容的提问来源于stack exchange，提问作者jthibaut