如何基于class包的KNN算法结合ROCR包获取AUC值？

Got it, let's break down why you're hitting this issue and how to fix it. The core problem is that knn() from the class package works differently than your classification tree model: it returns hard class labels by default, not the probability scores that ROCR's prediction() function requires. Unlike tree models (where you have a trained model object to call predict() on for probabilities), KNN is a lazy learner—there's no "trained model" to reuse, so you need to capture confidence scores when you first run the knn() function.

Here's the step-by-step solution:

1. Capture Confidence Scores from KNN #

When you call knn(), add the prob=TRUE parameter. This attaches an attribute to the output containing the proportion of nearest neighbors that voted for the predicted class (this acts as your confidence score, which we'll use as a proxy for probability):

# Run KNN and capture prediction confidence
knn_pred <- knn(train, test, train$Digit, k=1, prob=TRUE)

2. Format Scores for ROCR #

ROCR needs a continuous score (like a probability) aligned with your target class. Assuming you're working with a binary classification problem (matching your tree workflow where you pulled [,2] from the probability matrix):

• First extract the confidence scores from the knn_pred object
• Adjust scores to reflect the probability of your positive class (e.g., if your positive class is "1", use the confidence score if the prediction is "1", or 1 - confidence if it's the other class)

Example code:

# Extract the confidence scores (proportion of neighbors for the predicted class)
knn_confidence <- attr(knn_pred, "prob")

# Define your positive target class (replace with your actual positive label)
positive_class <- "1"

# Adjust scores to represent the probability of the positive class
knn_positive_probs <- ifelse(knn_pred == positive_class, knn_confidence, 1 - knn_confidence)

3. Calculate AUC with ROCR #

Now use these adjusted probability scores with the same ROCR workflow you used for the classification tree:

# Create ROCR prediction object
Pred <- prediction(knn_positive_probs, test$Digit)

# Calculate AUC performance
Perf <- performance(Pred, "auc")

# Extract the final AUC value
Perf@y.values[[1]]

Why Your Original Approach Failed #

You tried calling predict(knn_one, test, ...) but knn_one is just a vector of class labels, not a trained model object. KNN doesn't produce a reusable model like classification trees do—every prediction relies directly on comparing test data to the training set, so you have to capture confidence scores in the initial knn() call.

For multi-class problems, extend this with a one-vs-rest strategy: calculate AUC for each class individually by treating it as the positive class and all others as negative, then average the results if needed.

内容的提问来源于stack exchange，提问作者The Statistician Magician