confusion matrix构建疑问：Ranger与caret包实现的困惑

Hey there, let's break this down step by step to resolve your confusion about building and interpreting confusion matrices with caret::confusionMatrix!

First: The Critical Parameter Order in confusionMatrix #

The biggest issue here is the order of inputs to both table() and confusionMatrix() itself.

caret::confusionMatrix expects:

• The first argument to be your predicted values
• The second argument to be your reference (true) labels

So when you create the table, it should be structured as table(predicted_values, true_labels).

Why Your Two Calls Give Different Results #

When you run:

confusionMatrix(table(Test_Set$Species, max.col(probabilities)-1))

You’re passing true_labels first and predicted_values second to table(). This swaps the rows and columns of the confusion matrix, which in turn flips how tp, tn, fp, and fn are calculated. Since sensitivity, specificity, PPV, and NPV all depend on these four values, the metrics end up referring to the wrong class or being inverted.

On the other hand, this call is aligned with caret's expectations:

caret::confusionMatrix(table(max.col(probabilities) - 1, Test_Set$Species), positive = '1')

Here, predicted values are first, true labels are second, and you explicitly define which class is positive—this is the correct approach.

Verifying Your Confusion Matrix Values #

You mentioned that specifying positive = '1' gives you:

• tp=13, tn=36, fp=0, fn=1

Let’s confirm what these mean (with positive class = 1):

• tp (True Positive): 13 samples where the true label was 1 and your model correctly predicted 1
• tn (True Negative): 36 samples where the true label was 0 and your model correctly predicted 0
• fp (False Positive): 0 samples where the true label was 0 but the model incorrectly predicted 1
• fn (False Negative): 1 sample where the true label was 1 but the model incorrectly predicted 0

These values are correctly calculated if your model’s predictions and test set labels align with this breakdown. The key is that by setting positive='1' and passing predictions first, caret uses the right positions in the matrix to count these metrics.

Quick Refresher on Metric Interpretations #

To make sure you’re interpreting these correctly for your positive class (1):

• Sensitivity (Recall): tp/(tp+fn) = 13/(13+1) ≈ 92.86% → Percentage of actual positive cases your model correctly identified
• Specificity: tn/(tn+fp) = 36/(36+0) = 100% → Percentage of actual negative cases your model correctly identified
• PPV (Precision): tp/(tp+fp) = 13/(13+0) = 100% → Percentage of predicted positive cases that are actually positive
• NPV: tn/(tn+fn) = 36/(36+1) ≈ 97.3% → Percentage of predicted negative cases that are actually negative

Key Takeaways #

• Always structure your table() as table(predicted, true) when passing to confusionMatrix()
• Never skip the positive parameter—this ensures caret calculates metrics for the class you care about, not just the first factor level
• Double-check the matrix rows (predictions) and columns (true labels) to confirm tp/tn/fp/fn counts match your expectations

内容的提问来源于stack exchange，提问作者Ray