如何在Keras中修改预训练ResNet的conv5模块以复现DeeperCut？

数据老陈

2026-5-6

可以用预训练ResNet实现，无需从头训练整个网络！

首先明确：预训练ResNet的层属性（比如步长、空洞率）创建后无法直接修改，但我们可以复用conv1到conv4的预训练权重，重新构建符合要求的conv5层，这样既保留了预训练的特征提取能力，又满足DeeperCut的结构要求。

核心修改思路对应论文要求

将conv5组的第一个卷积层步长从2px改为1px以防止下采样；为conv5中所有3x3卷积添加空洞以保持感受野，使整个CNN的步长降至16px。

原来的ResNet50总步长是32（conv1到conv5各层下采样累计），修改conv5后，总步长变为16（conv1到conv4的下采样累计为16，conv5不再下采样）。
用dilation_rate=(2,2)的空洞卷积替代原来的步长2卷积，这样能保持和原步长2卷积相近的感受野，同时避免下采样。

Keras中的具体操作步骤

1. 加载预训练ResNet并提取conv4的输出

首先加载预训练ResNet50，拿到conv4的最后输出层（这部分我们完全复用预训练权重，不做修改）：

from tensorflow.keras.applications import ResNet50
from tensorflow.keras.layers import Conv2D, BatchNormalization, Activation, Add, Conv2DTranspose
from tensorflow.keras.regularizers import l2
from tensorflow.keras.models import Model

# 加载预训练ResNet50，不包含顶层
base_model = ResNet50(include_top=False, weights='imagenet')
# 获取conv4的最终输出层（ResNet50中该层名称为'conv4_block6_out'）
conv4_output = base_model.get_layer('conv4_block6_out').output

2. 重新构建修改后的conv5组

我们手动构建conv5的三个残差块，修改步长并添加空洞卷积，同时复用原预训练的权重（保证特征提取的连续性）：

# -------------------------- 构建conv5_block1 --------------------------
# 原conv5_block1_conv1步长为2，改为1，添加dilation_rate=(2,2)
x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block1_conv1').get_weights())(conv4_output)
x = BatchNormalization(weights=base_model.get_layer('conv5_block1_bn1').get_weights())(x)
x = Activation('relu')(x)

x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block1_conv2').get_weights())(x)
x = BatchNormalization(weights=base_model.get_layer('conv5_block1_bn2').get_weights())(x)
x = Activation('relu')(x)

x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block1_conv3').get_weights())(x)
x = BatchNormalization(weights=base_model.get_layer('conv5_block1_bn3').get_weights())(x)

# 原shortcut是步长2的1x1卷积，现在改为步长1（不需要降采样）
shortcut = Conv2D(2048, (1,1), strides=(1,1), padding='same',
                  kernel_initializer='he_normal',
                  weights=base_model.get_layer('conv5_block1_conv1').get_weights())(conv4_output)
shortcut = BatchNormalization(weights=base_model.get_layer('conv5_block1_bn1').get_weights())(shortcut)

x = Add()([x, shortcut])
conv5_block1_out = Activation('relu')(x)

# -------------------------- 构建conv5_block2 --------------------------
x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block2_conv1').get_weights())(conv5_block1_out)
x = BatchNormalization(weights=base_model.get_layer('conv5_block2_bn1').get_weights())(x)
x = Activation('relu')(x)

x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block2_conv2').get_weights())(x)
x = BatchNormalization(weights=base_model.get_layer('conv5_block2_bn2').get_weights())(x)
x = Activation('relu')(x)

x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block2_conv3').get_weights())(x)
x = BatchNormalization(weights=base_model.get_layer('conv5_block2_bn3').get_weights())(x)

x = Add()([x, conv5_block1_out])
conv5_block2_out = Activation('relu')(x)

# -------------------------- 构建conv5_block3 --------------------------
x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block3_conv1').get_weights())(conv5_block2_out)
x = BatchNormalization(weights=base_model.get_layer('conv5_block3_bn1').get_weights())(x)
x = Activation('relu')(x)

x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block3_conv2').get_weights())(x)
x = BatchNormalization(weights=base_model.get_layer('conv5_block3_bn2').get_weights())(x)
x = Activation('relu')(x)

x = Conv2D(2048, (3,3), strides=(1,1), padding='same', dilation_rate=(2,2),
           kernel_initializer='he_normal',
           weights=base_model.get_layer('conv5_block3_conv3').get_weights())(x)
x = BatchNormalization(weights=base_model.get_layer('conv5_block3_bn3').get_weights())(x)

x = Add()([x, conv5_block2_out])
conv5_output = Activation('relu')(x)

3. 连接原有的转置卷积层并构建模型

把新的conv5输出接回你原来的转置卷积层，然后设置训练策略：

# 连接你原来的转置卷积层
x = Conv2DTranspose(1, kernel_size=[3, 3], strides=2, kernel_regularizer=l2(0.0001))(conv5_output)
pred = Conv2DTranspose(1, kernel_size=[3, 3], strides=2, kernel_regularizer=l2(0.0001))(x)

# 构建完整模型
model = Model(inputs=base_model.input, outputs=pred)

# 冻结conv1到conv4的预训练层，只训练修改后的conv5和顶层转置卷积
for layer in base_model.layers:
    # 原base_model中的conv5层我们已经替换，所以冻结所有非conv5的层
    if 'conv5' not in layer.name:
        layer.trainable = False

# 编译模型（注意：如果是单通道输出，categorical_crossentropy可能不合适，可根据任务调整为binary_crossentropy等）
model.compile(optimizer='adam', loss='categorical_crossentropy')

关键注意事项

空洞率的选择：用dilation_rate=(2,2)是因为原步长2的3x3卷积，感受野等价于步长1+空洞2的3x3卷积，完美贴合论文“保持感受野”的要求。
权重复用：我们直接加载原预训练conv5层的权重，避免从零开始训练这部分，大幅节省训练时间。
shortcut的修改：原conv5的shortcut是用来匹配下采样后的特征图尺寸，现在步长改为1，所以shortcut的步长也要改为1，不需要降采样。
微调策略：如果训练后期效果不佳，可以解冻conv4的部分层进行微调，进一步提升性能。

内容的提问来源于stack exchange，提问作者Jan