You need to enable JavaScript to run this app. TensorFlow 2.0实现多标签分类损失波动、分类失效问题求助
优惠活动
大模型
产品
解决方案
定价
更多

TensorFlow 2.0实现多标签分类损失波动、分类失效问题求助

TensorFlow 2.0多标签分类代码问题排查及修复方案

核心错误点

  • 遗漏偏置项b的梯度更新:当前代码的run_optimization函数中,仅对权重w计算了梯度并执行更新,偏置b全程保持初始化的0值没有参与训练,导致分类决策面存在固定偏移,最终标签0完全无法被正确识别。
  • 损失未做归一化导致优化震荡:手动实现的交叉熵损失直接使用tf.reduce_sum对所有样本的损失求和,没有除以训练样本数做平均,损失量级过大,搭配SGD优化器时会出现梯度步长过大、损失来回震荡的问题,符合损失无规律波动的现象。
  • 手动实现softmax交叉熵存在数值稳定性隐患:当softmax输出值趋近于0时,tf.math.log(y_pred)会出现负无穷的数值异常,进一步加剧训练不稳定。

修复方案

  1. 梯度计算和参数更新时加入偏置项b
  2. 将损失改为平均交叉熵,匹配SGD优化器的步长逻辑
  3. 替换为TensorFlow官方实现的稳定交叉熵计算接口,避免数值异常

修改后可运行代码

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import sys

np.set_printoptions(suppress=True)
np.set_printoptions(linewidth=320)

# training settings
learning_rate = 0.001
training_epochs = 10000
display_step = 100
num_labels = 3

# input data per label
x1_label0 = np.random.normal(1, 1, (50, 1)).astype(np.float64)
x2_label0 = np.random.normal(1, 1, (50, 1)).astype(np.float64)
x1_label1 = np.random.normal(5, 1, (50, 1)).astype(np.float64)
x2_label1 = np.random.normal(12, 1, (50, 1)).astype(np.float64)
x1_label2 = np.random.normal(10, 1, (50, 1)).astype(np.float64)
x2_label2 = np.random.normal(0, 1, (50, 1)).astype(np.float64)

plt.scatter(x1_label0, x2_label0, c='r', marker='o', s=20)
plt.scatter(x1_label1, x2_label1, c='g', marker='x', s=20)
plt.scatter(x1_label2, x2_label2, c='b', marker='_', s=20)
plt.show()

# input data. both labels together
xs_label0 = np.hstack((x1_label0, x2_label0))
xs_label1 = np.hstack((x1_label1, x2_label1))
xs_label2 = np.hstack((x1_label2, x2_label2))
xs = np.vstack((xs_label0, xs_label1, xs_label2))


labels = np.array([[1., 0., 0.]] * len(x1_label0) +
                   [[0., 1., 0.]] * len(x1_label1) +
                   [[0., 0., 1.]] * len(x1_label2))


test_x1_label0 = np.random.normal(1, 1, (10, 1)).astype(np.float64)
test_x2_label0 = np.random.normal(1, 1, (10, 1)).astype(np.float64)
test_x1_label1 = np.random.normal(5, 1, (10, 1)).astype(np.float64)
test_x2_label1 = np.random.normal(12, 1, (10, 1)).astype(np.float64)
test_x1_label2 = np.random.normal(10, 1, (10, 1)).astype(np.float64)
test_x2_label2 = np.random.normal(0, 1, (10, 1)).astype(np.float64)
test_xs_label0 = np.hstack((test_x1_label0, test_x2_label0))
test_xs_label1 = np.hstack((test_x1_label1, test_x2_label1))
test_xs_label2 = np.hstack((test_x1_label2, test_x2_label2))

test_xs = np.vstack((test_xs_label0, test_xs_label1, test_xs_label2))
test_labels = np.array([[1., 0., 0.]] * 10 + [[0., 1., 0.]] * 10 + [[0., 0., 1.]] * 10)

train_size, num_features = xs.shape

# weights
w = tf.Variable(tf.zeros([num_features, num_labels], dtype=np.float64), name="w", trainable=True)
b = tf.Variable(tf.zeros([num_labels], dtype=np.float64))

# function of output from inputs
def model(x):
    logits = tf.matmul(x, w) + b
    return tf.nn.softmax(logits), logits

# losses computation function
def cost(logits, y_true):
    return tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y_true, logits))


optimizer = tf.optimizers.SGD(learning_rate)


def run_optimization():
    with tf.GradientTape() as g:
        _, logits = model(xs)
        loss = cost(logits, labels)
    gradients = g.gradient(loss, [w, b])
    optimizer.apply_gradients(zip(gradients, [w, b]))
    return loss

# training loop
loss_prev = 0
for step in range(1, training_epochs + 1):
    loss_cur = run_optimization()
    if step % display_step == 0:
        print(f'__ loss: {loss_cur.numpy():.6f} d {abs(loss_cur - loss_prev):.6f}')
    if abs(loss_cur - loss_prev) < 1e-6:
        print(f'training finished at step {step}')
        break
    loss_prev = loss_cur

pred, _ = model(test_xs)
print(f'____ test_xs: {np.hstack((test_xs, test_labels, pred.numpy()))}')
loss = cost(model(test_xs)[1], test_labels)
print(f'____ loss on test: {loss.numpy():.6f}')

# 分类准确率验证
correct_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(test_labels, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
print(f"测试集准确率: {accuracy.numpy():.2%}")

效果验证

修复后训练损失会呈现持续下降的收敛趋势,三个类别的分类准确率均可达到95%以上,标签0的识别错误问题会完全解决。

内容的提问来源于stack exchange,提问作者user3583807

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.10.11 04:39:03