tf.nn.softmax_cross_entropy_with_logits_v2返回0的三层MLP二分类问题

Hey, let's break down why you're seeing that weird zero loss and how to fix it, based on your dataset setup and model details:

1. You're using a multi-class loss function for binary classification #

tf.nn.softmax_cross_entropy_with_logits_v2 is built for multi-class tasks (3+ categories), but you're doing binary classification. While you can force it to work (by outputting 2 logits), it's far easier to use the right tool for the job: tf.nn.sigmoid_cross_entropy_with_logits.

If you insist on using softmax, your model's output layer needs 2 neurons (one for each class), and your labels (y_train/y_test) need to be one-hot encoded (shaped like (150,2) instead of (150,1)). Using single-dimensional 0/1 labels with softmax is a common mistake that leads to zero loss.

2. Label and logit shapes don't match #

Here's the key rule for softmax_cross_entropy_with_logits_v2:

• Logits must be shaped [batch_size, num_classes]
• Labels must match that shape (one-hot encoded)

Your y_train is (150,1), and if your model outputs 1 neuron (common for binary tasks), your logits are (150,1). Softmax on a single dimension always outputs 1.0, so when you calculate cross entropy:

If your label is 1: -1 * log(1.0) = 0
If your label is 0: -0 * log(1.0) = 0

That's why you're getting zero loss—total mismatch between your loss function choice and your model/label setup.

Quick fixes:

• Option 1 (Switch to binary loss): Keep your output layer as 1 neuron, replace the loss function with tf.nn.sigmoid_cross_entropy_with_logits (your existing (150,1) labels work perfectly here).
• Option 2 (Stick with softmax): Change your output layer to 2 neurons, then convert your labels to one-hot with tf.one_hot(tf.squeeze(y_train), depth=2).

3. Your batch size is too big #

Your training set only has 150 samples, but you set batch_size=200. That means every training step uses the entire dataset as one batch. While this isn't always a problem, it can lead to unexpected behavior (like the model fitting the tiny dataset instantly, or TensorFlow handling the batch dimension weirdly). Drop the batch size to something reasonable, like 32 or 64, to avoid edge cases.

4. Feature overload is causing unstable training #

You have 1929 features (thanks to one-hot encoding) but only 150 training samples—this is a classic case of more features than data. Your model can easily overfit, but it can also produce extreme logit values (from poorly initialized weights) that make softmax output 0 or 1, leading to zero cross entropy.

Fixes for this:

• Normalize your input features: Even one-hot features are 0/1, but any other features should be scaled to 0 mean and 1 variance to stabilize training.
• Use better weight initialization: Swap default initializers for something like tf.keras.initializers.GlorotUniform() to avoid extreme initial logits.
• Add regularization: Throw in L2 regularization on your dense layers or a Dropout layer between hidden layers to prevent overfitting and extreme outputs.

5. Double-check your loss calculation code #

Make sure you're not making these common coding mistakes:

• Don't apply softmax to your logits before passing them to the loss function—the function does this internally, and doing it twice will break the calculation.
• Verify your labels are strictly 0 or 1 (no typos like 2s or NaNs).
• Check that you're averaging the loss correctly (using tf.reduce_mean instead of a wrong sum/divide combo).

Example Code Snippets #

Correct Binary Classification with Sigmoid Loss

# Output layer: 1 neuron for binary classification
logits = tf.layers.dense(last_hidden_layer, units=1)
# Loss calculation (uses existing (150,1) labels)
loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(labels=y_train, logits=logits))

Correct Binary Classification with Softmax (Not Recommended, But Possible)

# Output layer: 2 neurons for two classes
logits = tf.layers.dense(last_hidden_layer, units=2)
# Convert labels to one-hot (from (150,1) to (150,2))
y_train_onehot = tf.one_hot(tf.squeeze(y_train), depth=2)
# Loss calculation
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=y_train_onehot, logits=logits))

内容的提问来源于stack exchange，提问作者Andrew Davidson