基于ResNet18的卷积自编码器输出全黑图像问题求助

我在Adaptiope数据集上训练自编码器，编码器采用ResNet18作为backbone，但训练多轮后重建图像始终为全黑。改用无ResNet18的简单编码器时，重建效果符合预期。参数设置为：num_input_channels: 224，base_channel_size: 3，latent_dim: 64。推测是架构问题但无法定位，求帮忙排查。

核心问题排查及修复方向 #

1. ResNet18输入通道不匹配：
   预训练ResNet18默认仅支持3通道输入，但你的数据集输入是224通道，直接使用会导致特征提取完全失效，必须修改ResNet18的第一层卷积来适配输入通道数。

2. 过度冻结ResNet参数：
   你将ResNet18的所有参数都冻结（param.requires_grad = False），仅靠后续浅层全连接层无法学习到足够的重建特征。建议解冻ResNet的后几层残差块，保留部分预训练特征的同时允许模型适配当前任务。

3. 解码器维度不匹配：
   解码器中硬编码的维度（如351232、28x28特征图）与编码器输出的latent vector维度不匹配，导致卷积转置层无法正确还原图像，需重新计算并对齐维度。

4. 输出通道与数据集不匹配：
   解码器最后一层卷积转置的输出通道是3，但你的数据集输入是224通道，这会导致输出维度错误，必须改为num_input_channels。

各模块代码 #

无ResNet18的编码器（Vanilla Encoder） #

class Encoder(nn.Module):

    def __init__(self,
                 num_input_channels : int,
                 base_channel_size : int,
                 latent_dim : int
                 ):
        """
        Inputs:
            - num_input_channels : Number of input channels of the image. For CIFAR, this parameter is 3
            - base_channel_size : Number of channels we use in the first convolutional layers. Deeper layers might use a duplicate of it.
            - latent_dim : Dimensionality of latent representation z
            - act_fn : Activation function used throughout the encoder network
        """
        super().__init__()
        c_hid = base_channel_size
        self.layer1 = nn.Sequential(nn.Conv2d(num_input_channels, c_hid, kernel_size=3, padding=1, stride=2), # 32x32 => 16x16
                    nn.ReLU(),
                    nn.Conv2d(c_hid, c_hid, kernel_size=3, padding=1),
                    nn.ReLU(),
                    nn.Conv2d(c_hid, 2*c_hid, kernel_size=3, padding=1, stride=2), # 16x16 => 8x8
                    nn.ReLU(),
                    nn.Conv2d(2*c_hid, 2*c_hid, kernel_size=3, padding=1),
                    nn.ReLU(),
                    nn.Conv2d(2*c_hid, 2*c_hid, kernel_size=3, padding=1, stride=2), # 8x8 => 4x4
                    nn.ReLU(),
                    nn.Flatten(), # Image grid to single feature vector
                    nn.Linear(351232, latent_dim))
        
        self.linear2 = nn.Linear(latent_dim, 20*8)
        self.softmax = nn.Softmax(dim=-1)

    def forward(self, x):
        enc = self.layer1(x)
        lin_p = self.linear2(enc) 
        p = self.softmax(lin_p)
        return enc, p 

带ResNet18的编码器（修复版） #

class Encoder(nn.Module):

    def __init__(self,
                 num_input_channels : int,
                 base_channel_size : int,
                 latent_dim : int
                 ):
        super().__init__()
        self.fc_hidden1 = 224

        # 加载预训练ResNet18并修改输入通道
        model = models.resnet18(pretrained=True)
        # 替换第一层卷积，适配224输入通道
        model.conv1 = nn.Conv2d(num_input_channels, 64, kernel_size=(7,7), stride=(2,2), padding=(3,3), bias=False)
        # 冻结前两层残差块，解冻后两层用于任务适配
        for name, param in model.named_parameters():
            if "layer3" not in name and "layer4" not in name:
                param.requires_grad = False

        modules = list(model.children())[:-1]
        self.resnet = nn.Sequential(*modules)
        self.fc1 = nn.Linear(model.fc.in_features, self.fc_hidden1) 
        self.bn1 = nn.BatchNorm1d(self.fc_hidden1, momentum=0.01)
        self.relu = nn.ReLU(inplace=True)
        self.layer = nn.Sequential(
            nn.Flatten(),
            nn.Linear(self.fc_hidden1, latent_dim)
        )
        
        self.linear2 = nn.Linear(latent_dim, 20*8) 
        self.softmax = nn.Softmax(dim=-1)

    def forward(self, x):
        x = self.resnet(x)
        x = x.flatten(start_dim=1)  # 替代手动reshape，适配动态维度
        x = self.fc1(x)
        x = self.bn1(x)
        x = self.relu(x)
        enc = self.layer(x)
        
        lin_p = self.linear2(enc) 
        p = self.softmax(lin_p)
        return enc, p 

共享解码器（修复版） #

class Decoder_N(nn.Module):

    def __init__(self,
                 num_input_channels : int,
                 base_channel_size : int,
                 latent_dim : int,
                 act_fn : object = nn.GELU):
        super().__init__()
        c_hid = 224
        # 计算匹配的线性层输出维度：对应4x4x(2*c_hid)特征图
        linear_out_dim = 2 * c_hid * 4 * 4
        self.linear = nn.Sequential(
            nn.Linear(latent_dim, linear_out_dim),
            nn.ReLU()
        )
        self.net = nn.Sequential(
            nn.ConvTranspose2d(2*c_hid, 2*c_hid, kernel_size=3, output_padding=1, padding=1, stride=2), # 4x4 => 8x8
            nn.ReLU(),
            nn.Conv2d(2*c_hid, 2*c_hid, kernel_size=3, padding=1),
            nn.ReLU(),
            nn.ConvTranspose2d(2*c_hid, c_hid, kernel_size=3, output_padding=1, padding=1, stride=2), # 8x8 => 16x16
            nn.ReLU(),
            nn.Conv2d(c_hid, c_hid, kernel_size=3, padding=1),
            nn.ReLU(),
            nn.ConvTranspose2d(c_hid, num_input_channels, kernel_size=3, output_padding=1, padding=1, stride=2), # 16x16 => 32x32，适配224通道输出
            nn.Tanh()
        )

    def forward(self, x):
        x = self.linear(x)
        x = x.reshape(x.shape[0], 2*c_hid, 4, 4)  # 对齐线性层输出维度
        x = self.net(x)
        return x

内容的提问来源于stack exchange，提问作者afzde