多分类离散变量预测的精准度衡量指标（除classification accuracy外）

Great question—when you’re moving from binary to multi-class classification, it’s easy to feel like the familiar metrics fall short. Beyond overall classification accuracy, there are tons of targeted metrics that give you a much clearer picture of how your model performs, especially across different classes. Let’s break down the most useful ones:

These are extensions of the binary metrics you know, adapted to handle multiple classes. For each individual class, you can calculate:

• Precision: The proportion of samples predicted as this class that are actually members of the class. Formula: TP / (TP + FP) where TP is true positives for the class, and FP is samples from other classes incorrectly labeled as this one. Great for when you want to minimize false positives (e.g., avoiding mislabeling a rare disease as common).
• Recall (Sensitivity): The proportion of actual members of the class that were correctly identified. Formula: TP / (TP + FN) where FN is samples of this class mislabeled as other classes. Useful when you can’t afford to miss cases (e.g., detecting fraud).
• F1-Score: The harmonic mean of precision and recall, balancing both metrics: 2 * (Precision * Recall) / (Precision + Recall). Perfect when you need to balance false positives and false negatives for a specific class.

To get an overall view across all classes, you can average these metrics three ways:

• Macro-average: Calculate the metric for each class, then take the simple arithmetic mean. Ignores class imbalance, so best for datasets with roughly equal sample sizes per class.
• Micro-average: Sum all TP, FP, and FN across classes first, then compute the metric once. Gives equal weight to every sample, making it ideal for imbalanced datasets where you care about overall correct predictions.
• Weighted-average: Calculate the metric for each class, then average them weighted by the number of samples in each class. Balances the focus between large and small classes.

The multi-class confusion matrix is your first stop—it’s a table that shows how many samples from each class were predicted as every other class. It’s incredibly intuitive for spotting which classes your model confuses the most. From it, you can derive:

• Cohen’s Kappa: Adjusts overall accuracy by accounting for the chance of random correct predictions. Formula: (Observed Accuracy - Expected Accuracy) / (1 - Expected Accuracy). This is way more reliable than raw accuracy for imbalanced datasets or when comparing models against human annotators.
• Matthews Correlation Coefficient (MCC): A single score between -1 (perfectly wrong) and 1 (perfectly correct) that considers all elements of the confusion matrix. It’s especially useful for imbalanced multi-class problems because it doesn’t favor majority classes.

You can extend ROC curves to multi-class scenarios with two common strategies:

• One-vs-Rest (OvR) AUC: For each class, treat it as the "positive" class and all others as "negative," then compute a binary ROC curve and AUC score. You can average these AUCs (macro or micro) to get an overall metric. This is the most straightforward extension of binary ROC.
• One-vs-One (OvO) AUC: For every pair of classes, compute a binary ROC/AUC, then average all these scores. It’s more computationally expensive (since you have n*(n-1)/2 pairs for n classes) but can be more accurate for small datasets with distinct classes.

Unlike accuracy or F1, log loss looks at the probabilities your model outputs, not just the final class predictions. It measures how well the predicted probabilities align with the true labels: -sum(y_true * log(y_pred)) where y_true is one-hot encoded true labels and y_pred is the model’s predicted probabilities. A lower log loss means your model’s confidence matches the actual outcomes—great for cases where you need reliable probability estimates (e.g., medical diagnosis where confidence matters).

For single-label multi-class problems, this is essentially the proportion of samples misclassified (equivalent to 1 - Accuracy), but it shines for multi-label scenarios where a sample can belong to multiple classes. It calculates the average number of incorrect labels per sample, making it useful if you care about individual label errors rather than just overall class assignments.

Quick Guide to Choosing Metrics #

• Class-balanced datasets: Macro-average F1, Cohen’s Kappa, or OvR AUC.
• Imbalanced datasets: Micro-average F1, weighted F1, or MCC.
• Need probability estimates: Log Loss.
• Want to visualize errors: Multi-class confusion matrix.
• Compare to random chance: Cohen’s Kappa or MCC.

内容的提问来源于stack exchange，提问作者sma