使用Monte-Carlo Dropout时模型预测无方差的问题排查

我用Keras（PyTorch后端）开发图像分割CNN，基于UNET实现，尝试用预测阶段的Monte-Carlo Dropout来近似模型预测的不确定性。训练模型时没加Dropout，导出权重后在预测阶段重新加载，但对同一张自制的256×512灰度测试图多次预测时，明明设置了50%的Dropout率，结果却完全一致。我已经改用model.load_weights()而非加载整个模型，但问题还是没解决，请问怎么验证Dropout是否真正生效？

相关代码及测试逻辑如下：

drop_rate = 0.5
drop_train = True  # MC dropout at inference
# drop_train=False #normal (no) dropout at inference
# downsize the UNET for this example.
# the smaller network is faster to train
# and produces excellent results on the dataset at hand
nfilters = (N_filters / 8).astype("int")

# input
input_tensor = Input(shape=frames_test_set.shape[1:], name="input_tensor")

## Encoder
# Encoder block 0
e0 = Conv2D(filters=nfilters[0], kernel_size=(3, 3), padding="same")(input_tensor)
e0 = BatchNormalization(axis=batch_normalization_axis)(e0)
e0 = Activation("relu")(e0)
e0 = Conv2D(filters=nfilters[0], kernel_size=(3, 3), padding="same")(e0)
e0 = BatchNormalization(axis=batch_normalization_axis)(e0)
e0 = Activation("relu")(e0)

# Encoder block 1
e1 = MaxPooling2D((2, 2))(e0)
e1 = Conv2D(filters=nfilters[1], kernel_size=(3, 3), padding="same")(e1)
e1 = BatchNormalization(axis=batch_normalization_axis)(e1)
e1 = Activation("relu")(e1)
e1 = Conv2D(filters=nfilters[1], kernel_size=(3, 3), padding="same")(e1)
e1 = BatchNormalization(axis=batch_normalization_axis)(e1)
e1 = Activation("relu")(e1)

# Encoder block 2
e2 = Dropout(drop_rate)(e1, training=drop_train)
e2 = MaxPooling2D((2, 2))(e2)
e2 = Conv2D(filters=nfilters[2], kernel_size=(3, 3), padding="same")(e2)
e2 = BatchNormalization(axis=batch_normalization_axis)(e2)
e2 = Activation("relu")(e2)
e2 = Conv2D(filters=nfilters[2], kernel_size=(3, 3), padding="same")(e2)
e2 = BatchNormalization(axis=batch_normalization_axis)(e2)
e2 = Activation("relu")(e2)

# Encoder block 3
e3 = Dropout(drop_rate)(e2, training=drop_train)
e3 = MaxPooling2D((2, 2))(e3)
e3 = Conv2D(filters=nfilters[3], kernel_size=(3, 3), padding="same")(e3)
e3 = BatchNormalization(axis=batch_normalization_axis)(e3)
e3 = Activation("relu")(e3)
e3 = Conv2D(filters=nfilters[3], kernel_size=(3, 3), padding="same")(e3)
e3 = BatchNormalization(axis=batch_normalization_axis)(e3)
e3 = Activation("relu")(e3)

# Encoder block 4
e4 = Dropout(drop_rate)(e3, training=drop_train)
e4 = MaxPooling2D((2, 2))(e4)
e4 = Conv2D(filters=nfilters[4], kernel_size=(3, 3), padding="same")(e4)
e4 = BatchNormalization(axis=batch_normalization_axis)(e4)
e4 = Activation("relu")(e4)
e4 = Conv2D(filters=nfilters[4], kernel_size=(3, 3), padding="same")(e4)
e4 = BatchNormalization(axis=batch_normalization_axis)(e4)
e4 = Activation("relu")(e4)
# e4 = MaxPooling2D((2, 2))(e4)

## Decoder
# Decoder block 3
d3 = Dropout(drop_rate)(e4, training=drop_train)
d3 = UpSampling2D(
    (2, 2),
)(d3)
d3 = concatenate([e3, d3], axis=-1)  # skip connection
d3 = Conv2DTranspose(nfilters[3], (3, 3), padding="same")(d3)
d3 = BatchNormalization(axis=batch_normalization_axis)(d3)
d3 = Activation("relu")(d3)
d3 = Conv2DTranspose(nfilters[3], (3, 3), padding="same")(d3)
d3 = BatchNormalization(axis=batch_normalization_axis)(d3)
d3 = Activation("relu")(d3)

# Decoder block 2
d2 = Dropout(drop_rate)(d3, training=drop_train)
d2 = UpSampling2D(
    (2, 2),
)(d2)
d2 = concatenate([e2, d2], axis=-1)  # skip connection
d2 = Conv2DTranspose(nfilters[2], (3, 3), padding="same")(d2)
d2 = BatchNormalization(axis=batch_normalization_axis)(d2)
d2 = Activation("relu")(d2)
d2 = Conv2DTranspose(nfilters[2], (3, 3), padding="same")(d2)
d2 = BatchNormalization(axis=batch_normalization_axis)(d2)
d2 = Activation("relu")(d2)

# Decoder block 1
d1 = UpSampling2D(
    (2, 2),
)(d2)
d1 = concatenate([e1, d1], axis=-1)  # skip connection
d1 = Conv2DTranspose(nfilters[1], (3, 3), padding="same")(d1)
d1 = BatchNormalization(axis=batch_normalization_axis)(d1)
d1 = Activation("relu")(d1)
d1 = Conv2DTranspose(nfilters[1], (3, 3), padding="same")(d1)
d1 = BatchNormalization(axis=batch_normalization_axis)(d1)
d1 = Activation("relu")(d1)

# Decoder block 0
d0 = UpSampling2D(
    (2, 2),
)(d1)
d0 = concatenate([e0, d0], axis=-1)  # skip connection
d0 = Conv2DTranspose(nfilters[0], (3, 3), padding="same")(d0)
d0 = BatchNormalization(axis=batch_normalization_axis)(d0)
d0 = Activation("relu")(d0)
d0 = Conv2DTranspose(nfilters[0], (3, 3), padding="same")(d0)
d0 = BatchNormalization(axis=batch_normalization_axis)(d0)
d0 = Activation("relu")(d0)

# output
# out_class = Dense(1)(d0)
out_class = Conv2D(1, (1, 1), padding="same")(d0)
out_class = Activation("sigmoid", name="output")(out_class)

# create and compile the model
model = Model(inputs=input_tensor, outputs=out_class)
model.compile(
    loss={"output": "binary_crossentropy"},
    metrics={"output": "accuracy"},
    optimizer="adam",
)

model.load_weights(f"{filepath}.keras")
Y_ts_hat = model.predict(frames_test_set, batch_size=1)

T = 10

Y_ts_hat_variance = np.zeros(
    (Y_ts_hat.shape[0], Y_ts_hat.shape[1], Y_ts_hat.shape[2], 1, T)
)

Y_ts_hat_variance[:, :, :, :, 0] = Y_ts_hat

for t in range(T - 1):
    print(f"Model {t+1}/{T-1}")
    Y_ts_hat_variance[:, :, :, :, t + 1] = model.predict(frames_test_set, batch_size=1)

arrays_indentical = (
    Y_ts_hat_variance[25, :, :, 0, 0] == Y_ts_hat_variance[25, :, :, 0, -1]
).all()
print(f"Arrays identical: {arrays_indentical}")

我已经做了20次预测，对比首次和末次结果始终完全一致，正常MC Dropout生效的话应该有随机性差异。

一、先排查Dropout未生效的核心原因 #

1. training参数被predict()覆盖
   Keras的model.predict()默认会强制所有层进入推理模式（training=False），会覆盖你手动给Dropout层设置的training=drop_train。改用函数式调用模型，强制开启训练模式：

   # 替换原predict调用
   Y_ts_hat = model(frames_test_set, training=True)
   # 循环内同样修改
   Y_ts_hat_variance[:, :, :, :, t + 1] = model(frames_test_set, training=True)
   

   若不想影响BatchNorm层（保持其推理模式），可以单独控制Dropout层的状态：

   def mc_predict(model, x):
       for layer in model.layers:
           if isinstance(layer, Dropout):
               layer.training = True
           elif isinstance(layer, BatchNormalization):
               layer.training = False
       return model(x)
   

   之后用mc_predict(model, frames_test_set)替代predict()。

2. 随机种子固定导致掩码重复
   检查代码中是否固定了全局随机种子（如np.random.seed()、torch.manual_seed()），固定种子会让每次Dropout的掩码完全一致。可以在每次预测前重新设置不同种子：

   import torch
   import numpy as np
   
   for t in range(T - 1):
       print(f"Model {t+1}/{T-1}")
       torch.manual_seed(t)
       np.random.seed(t)
       Y_ts_hat_variance[:, :, :, :, t + 1] = model(frames_test_set, training=True)
   

二、验证Dropout是否生效的方法 #

1. 直接检查Dropout层输出
   创建中间模型提取Dropout层的输出，对比两次预测结果：

   # 替换index为你要观察的Dropout层在模型中的索引
   dropout_layer = model.get_layer(index=6)
   intermediate_model = Model(inputs=model.input, outputs=dropout_layer.output)
   
   out1 = intermediate_model(frames_test_set[25:26], training=True)
   out2 = intermediate_model(frames_test_set[25:26], training=True)
   
   # 返回False则说明Dropout生效
   print((out1 == out2).all())
   

2. 统计多次预测的方差
   计算同一像素点在多次预测中的方差，若方差大于0则Dropout生效：

   mean_pred = np.mean(Y_ts_hat_variance, axis=-1)
   var_pred = np.var(Y_ts_hat_variance, axis=-1)
   
   print(f"最大方差: {np.max(var_pred)}")
   print(f"最小方差: {np.min(var_pred)}")
   

3. 极端参数测试
   临时将drop_rate设为1.0，此时所有神经元会被丢弃，输出应接近全0。若两次预测结果有差异（或符合全0逻辑），说明Dropout层在工作。

内容的提问来源于stack exchange，提问作者ZebraEagle