TensorFlow自定义加权交叉熵损失函数触发TypeError求助

问题描述 #

使用TensorFlow解决多标签分类任务时，分别通过函数式和继承tf.keras.losses.Loss类实现自定义加权交叉熵损失函数，但训练模型时均触发TypeError，报错信息为：Expected int64, but got 0.875 of type 'float'。

相关代码 #

多标签分类神经网络代码 #

epoch_count = 100
batch_size = 512
hidden_units = 19
eta = 0.1

# Define the model architecture
model = keras.Sequential([
    keras.layers.Dense(X_train.shape[1], activation='relu', input_shape=(X_train.shape[1],)),
    keras.layers.Dense(hidden_units, activation='relu'),
    keras.layers.Dense(y_train.shape[1], activation='sigmoid')
])

# Compile the model
model.compile(optimizer='adam',
              loss=WeightedCrossEntropy(positive_weights=positive_weights, negative_weights=negative_weights),
              metrics=[
                  Precision(name='precision'),
                  Recall(name='recall'),
                  BinaryAccuracy(name='accuracy')
              ])

# Train the model
model.fit(X_train, y_train, epochs=epoch_count, batch_size=batch_size, shuffle=True)

函数式自定义损失函数代码 #

positive_weights = {}
negative_weights = {}
for c in label_vector[:, 0].astype(int):
    positive_weights[c] = merged_data.shape[0]/(2*np.count_nonzero(merged_data[c] == 1))
    negative_weights[c] = merged_data.shape[0]/(2*np.count_nonzero(merged_data[c] == 0))
    
def loss_fn(y_true, y_pred):
        loss = 0.0
        loss -= (positive_weights[424]*y_true[0]*tf.math.log(y_pred[0]) + negative_weights[424]*(1-y_true[0])*tf.math.log(1-y_pred[0]))
        loss -= (positive_weights[7]*y_true[1]*tf.math.log(y_pred[1]) + negative_weights[7]*(1-y_true[1])*tf.math.log(1-y_pred[1]))
        loss -= (positive_weights[402]*y_true[2]*tf.math.log(y_pred[2]) + negative_weights[402]*(1-y_true[2])*tf.math.log(1-y_pred[2]))
    
        return loss

类式自定义损失函数代码 #

class WeightedCrossEntropy(tf.keras.losses.Loss):
    def __init__(self, positive_weights, negative_weights, epsilon=1e-7, name="weighted_cross_entropy", **kwargs):
        super().__init__(name=name, **kwargs)
        self.positive_weights = positive_weights
        self.negative_weights = negative_weights
        self.epsilon = epsilon

    def call(self, y_true, y_pred):
        y_pred = tf.clip_by_value(y_pred, self.epsilon, 1 - self.epsilon)
        loss =- (self.positive_weights[424] * y_true[0] * tf.math.log(y_pred[0]) + self.negative_weights[424] * (
                    1 - y_true[0]) * tf.math.log(1 - y_pred[0]))
        loss -= (self.positive_weights[7] * y_true[1] * tf.math.log(y_pred[1]) + self.negative_weights[7] * (
                    1 - y_true[1]) * tf.math.log(1 - y_pred[1]))
        loss -= (self.positive_weights[402] * y_true[2] * tf.math.log(y_pred[2]) + self.negative_weights[402] * (
                    1 - y_true[2]) * tf.math.log(1 - y_pred[2]))
        return tf.math.reduce_mean(loss)

    def get_config(self):
        config = {
            'positive_weights': self.positive_weights,
            'negative_weights': self.negative_weights,
            'epsilon': self.epsilon
        }
        base_config = super().get_config()
        return {**base_config, **config}

报错堆栈 #

Traceback (most recent call last):
  File "C:\Users\User\PycharmProjects\BuyOrBurnTensorflow\venv\Lib\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "C:\Users\User\AppData\Local\Temp\__autograph_generated_file9u5ymrga.py", line 15, in tf__train_function
    retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
    ^^^^^
  File "C:\Users\User\AppData\Local\Temp\__autograph_generated_file2x61h4sy.py", line 11, in tf__call
    loss = -(ag__.ld(self).positive_weights[424] * ag__.ld(y_true)[0] * ag__.converted_call(ag__.ld(tf).math.log, (ag__.ld(y_pred)[0],), None, fscope) + ag__.ld(self).negative_weights[424] * (1 - ag__.ld(y_true)[0]) * ag__.converted_call(ag__.ld(tf).math.log, (1 - ag__.ld(y_pred)[0],), None, fscope))
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
TypeError: in user code:

    File "C:\Users\User\PycharmProjects\BuyOrBurnTensorflow\venv\Lib\site-packages\keras\engine\training.py", line 1284, in train_function  *
        return step_function(self, iterator)
    File "C:\Users\User\PycharmProjects\BuyOrBurnTensorflow\main.py", line 167, in call  *
        loss =- (self.positive_weights[424] * y_true[0] * tf.math.log(y_pred[0]) + self.negative_weights[424] * (

    TypeError: Expected int64, but got 0.875 of type 'float'.

python-BaseException

Process finished with exit code -1073741510 (0xC000013A: interrupted by Ctrl+C)

问题根源 #

你在损失函数中使用y_true[0]、y_pred[0]的索引方式是错误的：TensorFlow中y_true和y_pred的形状为(batch_size, num_labels)，第一维度是批量大小，直接用整数索引会被当作批量维度的索引，而你传入的权重是浮点数，导致类型不匹配触发错误。同时硬编码标签索引的方式也不符合多标签损失的计算逻辑。

修复方案 #

1. 转换权重为TensorFlow张量 #

首先将字典格式的权重转换为与标签顺序对应的张量，确保类型为float32：

# 按模型输出的标签顺序排列权重
label_order = [424, 7, 402]
pos_weights = tf.convert_to_tensor([positive_weights[c] for c in label_order], dtype=tf.float32)
neg_weights = tf.convert_to_tensor([negative_weights[c] for c in label_order], dtype=tf.float32)

2. 修复类式自定义损失函数 #

class WeightedCrossEntropy(tf.keras.losses.Loss):
    def __init__(self, pos_weights, neg_weights, epsilon=1e-7, name="weighted_cross_entropy", **kwargs):
        super().__init__(name=name, **kwargs)
        self.pos_weights = pos_weights
        self.neg_weights = neg_weights
        self.epsilon = epsilon

    def call(self, y_true, y_pred):
        y_pred = tf.clip_by_value(y_pred, self.epsilon, 1 - self.epsilon)
        # 对每个标签计算加权交叉熵
        loss_pos = self.pos_weights * y_true * tf.math.log(y_pred)
        loss_neg = self.neg_weights * (1 - y_true) * tf.math.log(1 - y_pred)
        # 对标签维度求和，再对批量样本取平均
        total_loss = -tf.reduce_sum(loss_pos + loss_neg, axis=1)
        return tf.reduce_mean(total_loss)

    def get_config(self):
        config = {
            'pos_weights': self.pos_weights,
            'neg_weights': self.neg_weights,
            'epsilon': self.epsilon
        }
        base_config = super().get_config()
        return {**base_config, **config}

3. 修复函数式自定义损失函数 #

def loss_fn(y_true, y_pred):
    y_pred = tf.clip_by_value(y_pred, 1e-7, 1 - 1e-7)
    loss_pos = pos_weights * y_true * tf.math.log(y_pred)
    loss_neg = neg_weights * (1 - y_true) * tf.math.log(1 - y_pred)
    total_loss = -tf.reduce_sum(loss_pos + loss_neg, axis=1)
    return tf.reduce_mean(total_loss)

4. 模型编译调整 #

使用修复后的损失函数：

# 类式版本
model.compile(optimizer='adam',
              loss=WeightedCrossEntropy(pos_weights=pos_weights, neg_weights=neg_weights),
              metrics=[
                  keras.metrics.Precision(name='precision'),
                  keras.metrics.Recall(name='recall'),
                  keras.metrics.BinaryAccuracy(name='accuracy')
              ])

# 或函数式版本
model.compile(optimizer='adam',
              loss=loss_fn,
              metrics=[
                  keras.metrics.Precision(name='precision'),
                  keras.metrics.Recall(name='recall'),
                  keras.metrics.BinaryAccuracy(name='accuracy')
              ])

补充说明 #

• 避免硬编码批量维度的索引，多标签损失应针对标签维度做广播运算，确保每个标签的加权逻辑正确。
• 保持权重张量与y_true、y_pred的数据类型一致，避免类型不匹配问题。
• 使用tf.reduce_sum对标签维度求和，再对批量取平均，符合多标签分类任务的损失计算逻辑。

内容的提问来源于stack exchange，提问作者tanzwood