PyTorch交叉熵DCA组合损失转换为Keras自定义多分类损失咨询

核心思路 #

该损失函数的设计参考医学影像分类领域的校准优化方案，核心是在交叉熵损失基础上增加预测置信度与准确率差值（DCA）作为辅助惩罚项，将置信度校准融入模型训练流程：当交叉熵持续下降但分类准确率停滞时，DCA项会自动施加惩罚，避免模型出现过度自信的置信度偏移问题。

原PyTorch版本实现 #

import torch
from torch.nn import functional as F

def cross_entropy_with_dca_loss(logits, labels, weights=None, alpha=1., beta=10.):        
    ce = F.cross_entropy(logits, labels, weight=weights)

    softmaxes = F.softmax(logits, dim=1)
    confidences, predictions = torch.max(softmaxes, 1)
    accuracies = predictions.eq(labels)
    mean_conf = confidences.float().mean()
    acc = accuracies.float().sum()/len(accuracies)
    dca = torch.abs(mean_conf-acc)
    loss = alpha*ce+beta*dca
    
    return loss

适配后Keras自定义损失函数 #

适配要求：输入为多分类one-hot真实标签y_true、softmax输出的预测概率y_pred，可直接替代categorical cross-entropy使用。

import tensorflow as tf
from tensorflow.keras import backend as K

def cross_entropy_with_dca_loss(alpha=1.0, beta=10.0, class_weights=None):
    """
    带DCA校准项的多分类损失函数
    :param alpha: 交叉熵损失权重，默认1.0
    :param beta: DCA校准项权重，默认10.0，可根据实际校准需求调整
    :param class_weights: 类别权重数组，可选，用于不平衡分类场景
    :return: 可直接传入model.compile的损失函数对象
    """
    def calculate_loss(y_true, y_pred):
        # 计算基础分类交叉熵损失
        ce = K.categorical_crossentropy(y_true, y_pred)
        # 若传入类别权重则加权处理
        if class_weights is not None:
            weight_tensor = K.constant(class_weights, dtype=K.floatx())
            sample_weights = K.gather(weight_tensor, K.argmax(y_true, axis=-1))
            ce = ce * sample_weights
        ce = K.mean(ce)

        # 计算DCA校准项
        confidences = K.max(y_pred, axis=-1)
        pred_labels = K.argmax(y_pred, axis=-1)
        real_labels = K.argmax(y_true, axis=-1)
        acc = K.mean(K.cast(K.equal(pred_labels, real_labels), K.floatx()))
        mean_conf = K.mean(confidences)
        dca = K.abs(mean_conf - acc)

        # 返回总损失
        return alpha * ce + beta * dca
    return calculate_loss

使用示例 #

# 模型编译时直接传入损失函数即可，可自定义alpha、beta参数
model.compile(
    optimizer='adam',
    loss=cross_entropy_with_dca_loss(alpha=1.0, beta=8.0),
    metrics=['accuracy']
)

内容的提问来源于stack exchange，提问作者shiva