如何从TensorFlow的Softmax层提取预测结果及相关指标？

Got it, let's work through your problem. You need to pull the Softmax layer's output (prediction probabilities) and calculate metrics like accuracy, while setting things up to add precision/recall/F1 later. Let's fix your code and add the functionality you need.

First, Let's Fix Key Issues in Your Original Code #

Before adding the new features, there are a couple of critical tweaks needed:

• Logits vs. Softmax: The tf.losses.softmax_cross_entropy function expects unactivated logits (not the Softmax output) as input—this avoids numerical instability from double-applying Softmax. So we'll split the final layer into raw logits and a separate Softmax calculation for probabilities.
• Accuracy Calculation: tf.metrics.accuracy requires initializing local variables (which your code misses). For quick validation, a simpler approach using tf.equal and tf.reduce_mean works better for one-off accuracy checks.
• Prediction Extraction: We need to explicitly get both the predicted class (argmax of Softmax outputs) and the raw probability values.

Modified Code with Required Features #

Here's the updated code that addresses all your needs:

import tensorflow as tf
from tensorflow.contrib.layers import fully_connected
from sklearn.datasets import fetch_mldata
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
import numpy as np

# Load and preprocess data
mnist = fetch_mldata('MNIST original', data_home="data/mnist/")
lb = LabelBinarizer().fit(mnist.target)
X_train, X_test, y_train, y_test = train_test_split(
    mnist.data, 
    lb.transform(mnist.target), 
    train_size=0.9, 
    test_size=0.1
)

# TensorFlow graph setup
X = tf.placeholder(tf.float32, shape=(None, 784))
y = tf.placeholder(tf.float32, shape=(None, 10))  # Match y's dtype to one-hot labels

# Hidden layer + raw logits (no activation)
lOne = fully_connected(inputs=X, num_outputs=100, activation_fn=tf.nn.elu)
logits = fully_connected(inputs=lOne, num_outputs=10, activation_fn=None)  # Raw logits

# Softmax output for prediction probabilities
pred_probs = tf.nn.softmax(logits)
# Predicted class (argmax of probabilities)
pred_class = tf.argmax(pred_probs, axis=1)
# True class (argmax of one-hot labels)
true_class = tf.argmax(y, axis=1)

# Metrics calculation
# Simple accuracy (no local variables needed)
accuracy = tf.reduce_mean(tf.cast(tf.equal(pred_class, true_class), tf.float32))
# Loss (using raw logits as intended)
loss = tf.losses.softmax_cross_entropy(logits=logits, onehot_labels=y)

# Optimizer
trainOP = tf.train.AdamOptimizer(0.001).minimize(loss)

# Batch setup
bSize = 100
batches = int(np.floor(X_train.shape[0]/bSize))  # Fixed batch count calculation

def batcher(dSet, bNum):
    return dSet[bSize*bNum : bSize*(bNum+1)]  # 0-based indexing is easier here

epochs = 2
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    
    for epoch in range(epochs):
        # Training loop
        total_train_loss = 0.0
        for batch in range(batches):
            X_batch = batcher(X_train, batch)
            y_batch = batcher(y_train, batch)
            _, batch_loss = sess.run([trainOP, loss], feed_dict={X: X_batch, y: y_batch})
            total_train_loss += batch_loss
        
        # Validation after each epoch
        test_loss, test_acc, test_probs, test_preds, test_trues = sess.run(
            [loss, accuracy, pred_probs, pred_class, true_class],
            feed_dict={X: X_test, y: y_test}
        )
        
        print(f"Epoch {epoch+1}/{epochs}")
        print(f"Train Loss: {total_train_loss/batches:.4f}")
        print(f"Test Loss: {test_loss:.4f}, Test Accuracy: {test_acc:.4f}")
        print(f"Sample Prediction Probabilities (first 3 samples):\n{test_probs[:3]}")
        print(f"Sample Predicted Classes: {test_preds[:3]}, True Classes: {test_trues[:3]}\n")

# No need to explicitly close the session—context manager handles it

Key Explanations: #

• Prediction Probabilities: pred_probs holds the full Softmax output for each sample (10 values per sample, summing to 1), which is exactly the prediction probabilities you need.
• Predicted Classes: pred_class uses tf.argmax to get the index of the highest probability (the predicted digit).
• Accuracy Calculation: We compare predicted classes to true classes (derived from one-hot labels) and take the mean for overall accuracy.
• Logits Fix: By using activation_fn=None for the final layer, we pass raw logits to the loss function, which is the recommended practice for numerical stability.

For Future Precision/Recall/F1 Calculation: #

When you're ready to add these metrics, you have two options:

1. TensorFlow Metrics: Use tf.metrics.precision and tf.metrics.recall (remember to initialize local variables with tf.local_variables_initializer() alongside the global ones).
2. Scikit-Learn Metrics: Since we're already extracting test_preds and test_trues as NumPy arrays, you can use sklearn.metrics.precision_score, recall_score, and f1_score directly after the session ends—this is often simpler for small-scale validation.

内容的提问来源于stack exchange，提问作者Salih F. Canpolat