FutureGAN训练出现通道数不匹配错误，无法使用RGB格式输入训练

核心报错信息 #

RuntimeError: Given groups=1, weight[512, 3, 1, 1, 1], so expected input[32, 1, 6, 4, 4] to have 3 channels, but got 1 channels instead

这类通道不匹配问题此前已有多次反馈，但现有解决方案均无法解决我的问题。我正在尝试复现FutureGAN模型，项目地址为https://github.com/TUM-LMF/FutureGAN。

复现步骤 #

我按照项目说明完成所有准备步骤后，执行以下命令启动训练：

python /content/FutureGAN/train.py --data_root='/content/mmnist-2/train'

训练脚本地址为https://github.com/TUM-LMF/FutureGAN/blob/master/train.py。
程序运行后完整报错如下：

... loading training configuration ... 
 ... saving training configuration to <_io.TextIOWrapper name='./logs/2021-08-24_172412/train_config.txt' mode='w' encoding='UTF-8'>
 ... creating initial models ... 
 ... initial models have been built successfully ... 
 ... saving initial model strutures to <_io.TextIOWrapper name='./logs/2021-08-24_172412/initial_model_structure_4x4.txt' mode='w' encoding='UTF-8'>
  0% 0/10140 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/content/FutureGAN/train.py", line 832, in <module>
    trainer.train()
  File "/content/FutureGAN/train.py", line 574, in train
    self.z_x_gen = self.G(self.z)
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/content/FutureGAN/model.py", line 307, in forward
    y = self.model(x)
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/modules/container.py", line 67, in forward
    input = module(input)
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/modules/container.py", line 67, in forward
    input = module(input)
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/content/FutureGAN/custom_layers.py", line 114, in forward
    x = self.conv(x.mul(self.scale))
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 388, in forward
    self.padding, self.dilation, self.groups)
  File "/usr/local/envs/FutureGAN/lib/python3.6/site-packages/torch/nn/functional.py", line 126, in conv3d
    return f(input, weight, bias)
RuntimeError: Given groups=1, weight[512, 3, 1, 1, 1], so expected input[32, 1, 6, 4, 4] to have 3 channels, but got 1 channels instead

已尝试的规避方案 #

我可以通过指定输入通道数为1的方式规避该问题，执行命令如下：

python /content/FutureGAN/train.py --data_root='/content/mmnist-2/train' --nc 1

但该方案不符合需求，我需要使用RGB格式的3通道输入进行训练。

运行环境与模型信息 #

运行环境为安装了Conda及所有依赖项的常规Google Colab环境，完整操作步骤见https://github.com/dhruvsheth-ai/FutureGAN/blob/master/FutureGAN-new.ipynb。
模型架构详情如下：

--------------------------------------------------
Sequences in Dataset:  16199
Global iteration step:  1014 , Epoch:  2
Phase:  init
Number of Generator`s model parameters: 
35659778
Number of Discriminator`s model parameters: 
66603521
--------------------------------------------------
New Generator structure: 
FutureGenerator(
  (model): Sequential(
    (concat_block_encode): Concat(
      (layer1): Sequential(
        (low_resl_from_rgb): Sequential(
          (from_rgb_block): Sequential(
            (0): EqualizedConv3d(
              (conv): Conv3d(1, 512, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)
            )
            (1): LeakyReLU(0.2)
            (2): PixelwiseNormLayer(
            )
          )
        )
        (low_resl_downsample): Sequential(
          (0): EqualizedConv3d(
            (conv): Conv3d(512, 512, kernel_size=(1, 2, 2), stride=(1, 2, 2), bias=False)
          )
          (1): LeakyReLU(0.2)
          (2): PixelwiseNormLayer(
          )
        )
      )
      (layer2): Sequential(
        (high_resl_from_rgb): Sequential(
          (0): EqualizedConv3d(
            (conv): Conv3d(1, 512, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)
          )
          (1): LeakyReLU(0.2)
          (2): PixelwiseNormLayer(
          )
        )
        (high_resl_block_encode): Sequential(
          (0): EqualizedConv3d(
            (conv): Conv3d(512, 512, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
          )
          (1): LeakyReLU(0.2)
          (2): PixelwiseNormLayer(
          )
          (3): EqualizedConv3d(
            (conv): Conv3d(512, 512, kernel_size=(1, 2, 2), stride=(1, 2, 2), bias=False)
          )
          (4): LeakyReLU(0.2)
          (5): PixelwiseNormLayer(
          )
        )
      )
    )
    (fadein_block_encode): FadeInLayer(
    )
    (middle_block): Sequential(
      (0): EqualizedConv3d(
        (conv): Conv3d(512, 512, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      )
      (1): LeakyReLU(0.2)
      (2): PixelwiseNormLayer(
      )
      (3): EqualizedConv3d(
        (conv): Conv3d(512, 512, kernel_size=(6, 1, 1), stride=(1, 1, 1), bias=False)
      )
      (4): LeakyReLU(0.2)
      (5): PixelwiseNormLayer(
      )
      (6): EqualizedConvTranspose3d(
        (deconv): ConvTranspose3d(512, 512, kernel_size=(6, 1, 1), stride=(1, 1, 1), bias=False)
      )
      (7): LeakyReLU(0.2)
      (8): PixelwiseNormLayer(
      )
      (9): EqualizedConv3d(
        (conv): Conv3d(512, 512, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      )
      (10): LeakyReLU(0.2)
      (11): PixelwiseNormLayer(
      )
    )
    (concat_block_decode): Concat(
      (layer1): Sequential(
        (low_resl_upsample): Sequential(
          (0): EqualizedConvTranspose3d(
            (deconv): ConvTranspose3d(512, 512, kernel_size=(1, 2, 2), stride=(1, 2, 2), bias=False)
          )
          (1): LeakyReLU(0.2)
          (2): PixelwiseNormLayer(
          )
        )
        (low_resl_to_rgb): Sequential(
          (to_rgb_block): Sequential(
            (0): EqualizedConv3d(
              (conv): Conv3d(512, 1, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)
            )
          )
        )
      )
      (layer2): Sequential(
        (high_resl_block_decode): Sequential(
          (0): EqualizedConvTranspose3d(
            (deconv): ConvTranspose3d(512, 512, kernel_size=(1, 2, 2), stride=(1, 2, 2), bias=False)
          )
          (1): LeakyReLU(0.2)
          (2): PixelwiseNormLayer(
          )
          (3): EqualizedConv3d(
            (conv): Conv3d(512, 512, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
          )
          (4): LeakyReLU(0.2)
          (5): PixelwiseNormLayer(
          )
        )
        (high_resl_to_rgb): Sequential(
          (0): EqualizedConv3d(
            (conv): Conv3d(512, 1, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)
          )
        )
      )
    )
    (fadein_block_decode): FadeInLayer(
    )
  )
)
--------------------------------------------------
New Discriminator structure: 
Discriminator(
  (model): Sequential(
    (concat_block): Concat(
      (layer1): Sequential(
        (low_resl_from_rgb): Sequential(
          (from_rgb_block): Sequential(
            (0): EqualizedConv3d(
              (conv): Conv3d(1, 512, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)
            )
            (1): LeakyReLU(0.2)
          )
        )
        (low_resl_downsample): Sequential(
          (0): EqualizedConv3d(
            (conv): Conv3d(512, 512, kernel_size=(1, 2, 2), stride=(1, 2, 2), bias=False)
          )
          (1): LeakyReLU(0.2)
        )
      )
      (layer2): Sequential(
        (high_resl_from_rgb): Sequential(
          (0): EqualizedConv3d(
            (conv): Conv3d(1, 512, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False)
          )
          (1): LeakyReLU(0.2)
        )
        (high_resl_block): Sequential(
          (0): EqualizedConv3d(
            (conv): Conv3d(512, 512, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
          )
          (1): LeakyReLU(0.2)
          (2): EqualizedConv3d(
            (conv): Conv3d(512, 512, kernel_size=(1, 2, 2), stride=(1, 2, 2), bias=False)
          )
          (3): LeakyReLU(0.2)
        )
      )
    )
    (fadein_block): FadeInLayer(
    )
    (last_block): Sequential(
      (0): MinibatchStdConcatLayer(averaging = all)
      (1): EqualizedConv3d(
        (conv): Conv3d(513, 512, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      )
      (2): LeakyReLU(0.2)
      (3): EqualizedConv3d(
        (conv): Conv3d(512, 512, kernel_size=(12, 4, 4), stride=(1, 1, 1), bias=False)
      )
      (4): LeakyReLU(0.2)
      (5): Flatten(
      )
      (6): EqualizedLinear(
        (linear): Linear(in_features=512, out_features=1, bias=False)
      )
    )
  )
)
--------------------------------------------------

该模型在包含RGB格式动作图像的KTH数据集上测试时同样出现该错误，恳请各位提供解决方案，若能给出具体代码修改建议将不胜感激。 #

内容的提问来源于stack exchange，提问作者Dhruv Sheth