GAN生成3D图像时出现输入维度不匹配错误，求解决方案

问题描述 #

此前成功训练GAN生成灰度图像，修改模型以生成3D图像时，在生成器中添加第三维度后触发报错，报错信息如下：

Traceback (most recent call last):
  File "/media/user/5EB3-54BF/gan3.py", line 84, in <module>
    generator = make_generator_model()
  File "/media/user/5EB3-54BF/gan3.py", line 63, in make_generator_model
    model.add(layers.Conv2DTranspose(256, (5, 5), strides=(1, 1), padding='same', use_bias=False))
  File "/home/user/.local/lib/python3.10/site-packages/tensorflow/python/trackable/base.py", line 205, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/user/.local/lib/python3.10/site-packages/keras/layers/convolutional/conv2d_transpose.py", line 205, in build
    raise ValueError(
ValueError: Inputs should have rank 4. Received input_shape=(None, 14, 14, 3, 512).

生成器代码：

def make_generator_model():
        model = tf.keras.Sequential() #make 14
        model.add(layers.Dense(14*14*3*512, use_bias=False, input_shape=(300,))) #ADD MORE NOISE!!!!!!!
        model.add(layers.BatchNormalization())
        model.add(layers.LeakyReLU())

        model.add(layers.Reshape((14, 14, 3, 512)))
        assert model.output_shape == (None, 14, 14, 3, 512)  # Note: None is the batch size

        model.add(layers.Conv2DTranspose(256, (5, 5), strides=(1, 1), padding='same', use_bias=False))
        assert model.output_shape == (None, 14, 14, 3, 256)
        model.add(layers.BatchNormalization())
        model.add(layers.LeakyReLU())

        model.add(layers.Conv2DTranspose(128, (5, 5), strides=(2, 2), padding='same', use_bias=False))
        assert model.output_shape == (None, 28, 28, 3, 128)
        model.add(layers.BatchNormalization())
        model.add(layers.LeakyReLU())
        
        #additional layer added here
        model.add(layers.Conv2DTranspose(64, (5, 5), strides=(2, 2), padding='same', use_bias=False))
        assert model.output_shape == (None, 56, 56, 3, 64)
        model.add(layers.BatchNormalization())
        model.add(layers.LeakyReLU())

        model.add(layers.Conv2DTranspose(1, (5, 5), strides=(2, 2), padding='same', use_bias=False, activation='tanh'))
        assert model.output_shape == (None, 112, 112, 3)

        return model

解决思路 #

核心问题 #

报错原因明确：你使用了2维转置卷积层（Conv2DTranspose），但输入的是5维张量（batch, H, W, D, channels），而Conv2DTranspose仅支持处理4维张量（batch, H, W, channels）。要生成3D图像，必须改用3维转置卷积层（Conv3DTranspose）。

具体修改步骤 #

1. 替换所有Conv2DTranspose为Conv3DTranspose
2. 调整卷积核形状：将原来的(5,5)改为(5,5,5)，对应3个空间维度的卷积核
3. 调整步幅参数：将原来的(1,1)/(2,2)改为(1,1,1)/(2,2,2)，确保三个空间维度同步放大（若需单独调整某维度步幅可按需修改）
4. 修正输出形状断言：3D图像最终输出应为5维张量（batch, H, W, D, 1），原代码最后一行的断言是2D RGB图像格式，需对应调整。

修改后的生成器代码 #

def make_generator_model():
        model = tf.keras.Sequential()
        model.add(layers.Dense(14*14*3*512, use_bias=False, input_shape=(300,)))
        model.add(layers.BatchNormalization())
        model.add(layers.LeakyReLU())

        model.add(layers.Reshape((14, 14, 3, 512)))
        assert model.output_shape == (None, 14, 14, 3, 512)  # (batch, H, W, D, channels)

        # 替换为Conv3DTranspose，卷积核和步幅改为3维
        model.add(layers.Conv3DTranspose(256, (5, 5, 5), strides=(1, 1, 1), padding='same', use_bias=False))
        assert model.output_shape == (None, 14, 14, 3, 256)
        model.add(layers.BatchNormalization())
        model.add(layers.LeakyReLU())

        model.add(layers.Conv3DTranspose(128, (5, 5, 5), strides=(2, 2, 2), padding='same', use_bias=False))
        assert model.output_shape == (None, 28, 28, 6, 128)  # D维度从3*2=6
        model.add(layers.BatchNormalization())
        model.add(layers.LeakyReLU())
        
        model.add(layers.Conv3DTranspose(64, (5, 5, 5), strides=(2, 2, 2), padding='same', use_bias=False))
        assert model.output_shape == (None, 56, 56, 12, 64)  # D维度6*2=12
        model.add(layers.BatchNormalization())
        model.add(layers.LeakyReLU())

        model.add(layers.Conv3DTranspose(1, (5, 5, 5), strides=(2, 2, 2), padding='same', use_bias=False, activation='tanh'))
        assert model.output_shape == (None, 112, 112, 24, 1)  # 最终3D图像形状：(batch, 112,112,24,1)

        return model

额外说明 #

• 若你的3D图像不需要在D维度上放大，可将对应层的strides中D维度设为1，比如strides=(2,2,1)，保持D维度不变，具体需根据数据集需求调整。
• 判别器部分也需同步修改为使用Conv3D层，保证与生成器的维度一致性。

内容的提问来源于stack exchange，提问作者Matthew Lukaszewski