TensorFlow Keras高阶API实现Logistic Regression结果异常求助

问题背景 #

用TensorFlow Keras低阶API实现的Logistic Regression结果与deeplearning.ai手动实现一致，逻辑正确，但使用高阶API实现时，损失值和准确率与低阶版本差异显著，结果异常。

低阶API代码（正确实现） #

## initialize part
print_cost=True
num_epochs = 10
learning_rate = 0.005

w = tf.Variable(tf.zeros(shape=(X.shape[0],1))) # shape=(2,1)
b = tf.Variable(0.0)

optimizer = tf.keras.optimizers.SGD(learning_rate)
accuracy = tf.keras.metrics.BinaryAccuracy()

## training part
for i in range(num_epochs):

    # forward pass
    with tf.GradientTape() as tape:
        z = tf.add(tf.matmul(tf.transpose(w), X), b)
        bce = tf.keras.losses.BinaryCrossentropy( from_logits=True, label_smoothing=0.0, axis=-1, reduction='sum', name='binary_crossentropy')
        cost = bce(Y, z)
    # backward pass
    trainable_variables = [w, b]
    grads = tape.gradient(cost, trainable_variables)
    optimizer.apply_gradients(zip(grads, trainable_variables))

    # Print the cost every training epoch
    if print_cost:
        accuracy.update_state(Y, tf.sigmoid(z))
        print(f"Epoch {i+1}/{num_epochs}  -  Cost:{cost.numpy()}  -  Accuracy:{accuracy.result().numpy()}")
        accuracy.reset_states()

低阶API运行结果 #

Epoch 1/10 - Cost:0.6931471824645996 - Accuracy:0.6555023789405823
Epoch 2/10 - Cost:0.7410295009613037 - Accuracy:0.6555023789405823
Epoch 3/10 - Cost:0.7531536817550659 - Accuracy:0.3444976210594177
Epoch 4/10 - Cost:0.8667086958885193 - Accuracy:0.6555023789405823
Epoch 5/10 - Cost:0.768563449382782 - Accuracy:0.34928229451179504
Epoch 6/10 - Cost:0.897014319896698 - Accuracy:0.6555023789405823
Epoch 7/10 - Cost:0.7556132674217224 - Accuracy:0.3684210479259491
Epoch 8/10 - Cost:0.8804366588592529 - Accuracy:0.6555023789405823
Epoch 9/10 - Cost:0.7514852285385132 - Accuracy:0.4019138813018799
Epoch 10/10 - Cost:0.8771329522132874 - Accuracy:0.6555023789405823

高阶API原代码（结果异常） #

# Create the model
model = k.models.Sequential()
model.add( k.layers.Dense(units=1, activation='sigmoid', input_shape=(X.shape[0],),     #X.shape[0]=2
                          kernel_initializer=k.initializers.Zeros(), bias_initializer=k.initializers.Zeros()) )

# Compile the model
model.compile(
    optimizer=k.optimizers.SGD(learning_rate=0.005),
    loss=k.losses.BinaryCrossentropy(),
    metrics=[k.metrics.BinaryAccuracy()],
)

# Train the model
model.fit(tf.transpose(X), tf.transpose(Y), epochs=10, shuffle=False, batch_size=1)

高阶API原运行结果 #

Epoch 1/10
209/209 [] - 1s 3ms/step - loss: 2.6512 - binary_accuracy: 0.5550
Epoch 2/10
209/209 [] - 1s 4ms/step - loss: 2.0159 - binary_accuracy: 0.6459
Epoch 3/10
209/209 [] - 1s 6ms/step - loss: 1.6735 - binary_accuracy: 0.6651
Epoch 4/10
209/209 [] - 1s 4ms/step - loss: 1.4703 - binary_accuracy: 0.6842
Epoch 5/10
209/209 [] - 1s 3ms/step - loss: 1.2202 - binary_accuracy: 0.7129
Epoch 6/10
209/209 [] - 1s 3ms/step - loss: 1.0095 - binary_accuracy: 0.7464
Epoch 7/10
209/209 [] - 1s 3ms/step - loss: 0.9206 - binary_accuracy: 0.7368
Epoch 8/10
209/209 [] - 1s 3ms/step - loss: 0.7307 - binary_accuracy: 0.7703
Epoch 9/10 209/209 [] - 0s 2ms/step - loss: 0.7977 - binary_accuracy: 0.7799
Epoch 10/10
209/209 [] - 0s 2ms/step - loss: 0.5660 - binary_accuracy: 0.7943

问题根源与修正方案 #

核心差异点 #

1. 损失函数输入逻辑：低阶API直接传入线性输出z并设置from_logits=True，高阶API却在Dense层加sigmoid激活，同时损失函数未对应调整，导致重复计算sigmoid，损失值错误。
2. 损失归约方式：低阶API用reduction='sum'计算总损失，高阶API默认用平均损失，数值量级不一致。
3. 训练批次：低阶API是全批次训练，高阶API用batch_size=1的随机梯度下降，优化路径完全不同。

修正后的高阶API代码 #

import tensorflow as tf
k = tf.keras

# 保持与低阶API一致的输入维度
X_input = X  # shape=(2,209)
Y_input = Y  # shape=(1,209)

# 创建模型：移除sigmoid激活，输出线性logits
model = k.models.Sequential()
model.add( k.layers.Dense(units=1, activation=None, input_shape=(X.shape[0],),
                          kernel_initializer=k.initializers.Zeros(), bias_initializer=k.initializers.Zeros()) )

# 编译模型：匹配低阶API的损失函数参数
model.compile(
    optimizer=k.optimizers.SGD(learning_rate=0.005),
    loss=k.losses.BinaryCrossentropy(from_logits=True, reduction='sum'),
    metrics=[k.metrics.BinaryAccuracy()],
)

# 训练模型：采用全批次训练，对齐低阶API的优化逻辑
model.fit(X_input, Y_input, epochs=10, shuffle=False, batch_size=X.shape[1])

修正说明 #

• 移除Dense层的sigmoid激活，让模型输出线性logits，同时损失函数设置from_logits=True，避免重复计算激活函数。
• 损失函数设置reduction='sum'，与低阶API的损失计算逻辑统一，保证数值量级一致。
• 设置batch_size=X.shape[1]，采用全批次训练，对齐低阶API的批量梯度下降优化路径。

内容的提问来源于stack exchange，提问作者zoraiz ramay