同一输入图像生成不同GradCam热力图的技术疑问

我尝试将GradCam应用于3D图像体的测试集以分析模型，代码基于keras.io的示例修改。但发现对同一图像调用make_gradcam_heatmap函数时，得到的热力图不同，且仅存在8种唯一结果——无论输入是癌/非癌测试集图像都是如此。

核心函数代码 #

def make_gradcam_heatmap(img_array, model, last_conv_layer_name, pred_index=None):
    # Generate class activation heatmap
    # First, we create a model that maps the input image to the activations
    # of the last conv layer as well as the output predictions
    grad_model = tf.keras.Model(
        [model.inputs], [model.get_layer(
            last_conv_layer_name).output, model.output]
    )

    # Then, we compute the gradient of the top predicted class for our input image
    # with respect to the activations of the last conv layer
    with tf.GradientTape() as tape:
        last_conv_layer_output, preds = grad_model(img_array)
        if pred_index is None:
            pred_index = tf.argmax(preds[0])
        class_channel = preds[:, pred_index]
        # print(class_channel)

    # This is the gradient of the output neuron (top predicted or chosen)
    # with regard to the output feature map of the last conv layer
    grads = tape.gradient(class_channel, last_conv_layer_output)

    # This is a vector where each entry is the mean intensity of the gradient
    # over a specific feature map channel (equivalent to global average pooling)
    pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2, 3))

    # We multiply each channel in the feature map array
    # by 'how important this channel is' with regard to the top predicted class
    # then sum all the channels to obtain the heatmap class activation
    last_conv_layer_output = last_conv_layer_output[0]
    heatmap = last_conv_layer_output @ pooled_grads[..., tf.newaxis]
    heatmap = tf.squeeze(heatmap)

    # For visualization purpose, we will also normalize the heatmap between 0 & 1
    # Notice that we clip the heatmap values, which is equivalent to applying ReLU
    heatmap = tf.maximum(heatmap, 0) / tf.math.reduce_max(heatmap)
    return heatmap.numpy()

def HeatmapGen(input_volume, model, last_conv_layer_name,pred_index = None):
    # Remove last layer's activation
    model.layers[-1].activation = None

    # Print what the top predicted class is
    img_array = np.expand_dims(input_volume, axis=0)

    # preds = model.predict(img_array)
    # print('Predicted:', preds[0])

    # Generate class activation heatmap
    heatmap = make_gradcam_heatmap(img_array, model, last_conv_layer_name, pred_index = pred_index)
    return heatmap

测试现象 #

测试代码执行后显示：每次调用grad_model(img_array)时，last_conv_layer_output和preds都会变化；多次测试仅得到形状为(8, 19, 46, 30)的8种唯一热力图结果。

模型结构 #

Model: "CanNonCan_v1.0.1_Model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 InputLayer (InputLayer)     [(None, 45, 99, 67, 1)]   0         
                                                                 
 Flip3DLayer (Flip3D)        (None, 45, 99, 67, 1)     0         
                                                                 
 3DConv1 (Conv3D)            (None, 44, 98, 66, 8)     72        
                                                                 
 3DConv2 (Conv3D)            (None, 43, 97, 65, 8)     520       
                                                                 
 MaxPool1 (MaxPooling3D)     (None, 21, 48, 32, 8)     0         
                                                                 
 BatchNorm1 (BatchNormalizat  (None, 21, 48, 32, 8)    32        
 ion)                                                            
                                                                 
 3DConv3 (Conv3D)            (None, 20, 47, 31, 16)    1040      
                                                                 
 3DConv4 (Conv3D)            (None, 19, 46, 30, 16)    2064      
                                                                 
 MaxPool2 (MaxPooling3D)     (None, 9, 23, 15, 16)     0         
                                                                 
 BatchNorm2 (BatchNormalizat  (None, 9, 23, 15, 16)    64        
 ion)                                                            
                                                                 
 GlobalNorm1 (GlobalAverageP  (None, 16)               0         
 ooling3D)                                                       
                                                                 
 FC1 (Dense)                 (None, 50)                850       
                                                                 
 FC2 (Dense)                 (None, 50)                2550      
                                                                 
 Dropout1 (Dropout)          (None, 50)                0         
                                                                 
 Classifier (Dense)          (None, 1)                 51        
                                                                 
=================================================================
Total params: 7,243
Trainable params: 7,195
Non-trainable params: 48
_________________________________________________________________

技术疑问 #

• 每个输入图像是否应生成唯一的GradCam热力图？
• 输入完全相同的情况下，为何preds在每次循环中都会变化？
• 为何仅存在8种唯一输出，且该数量与输入图像无关？

内容的提问来源于stack exchange，提问作者Joshua Arenson