线性最小二乘分类器实现疑问：精度评估与多类别扩展

Great question! Let's tackle your two problems one by one, with R code that builds on your existing implementation:

Your current code computes predictions for the training set, but to evaluate accuracy on the test set, you need to adjust how you calculate kernel values and compare predictions to true test labels. Here's how to do it:

Step-by-Step Implementation #

Assume you have x_test (test features) and y_test (true test labels, {-1,1}) ready.

a. Compute the test-train kernel matrix

Your existing gausskernel function calculates the kernel between training samples and themselves. For test data, we need a matrix where each entry K_test[i,j] is the Gaussian kernel between the i-th test sample and j-th training sample. Let's add a helper function for this:

# Helper to compute Gaussian kernel between two separate sample sets
gausskernel_two_sets <- function(X1, X2, sigma=1) {
  n1 <- nrow(X1)
  n2 <- nrow(X2)
  K <- matrix(0, nrow=n1, ncol=n2)
  for (i in 1:n1) {
    for (j in 1:n2) {
      K[i,j] <- exp(-sum((X1[i,] - X2[j,])^2)/(2*sigma^2))
    }
  }
  return(K)
}

# Calculate kernel matrix between test and training data
K_test <- gausskernel_two_sets(X1=x_test, X2=x_train, sigma=1)

b. Generate test set predictions

Skip the nested loops (they're slow in R!) and use matrix multiplication for efficiency:

# Compute f(x) for all test samples
f_test <- K_test %*% c

# Get predicted labels using sign()
y_pred <- sign(f_test)

c. Calculate accuracy

Use base R to compare predictions to true labels:

# Overall accuracy percentage
accuracy <- mean(y_pred == y_test)
cat("Test Set Accuracy:", round(accuracy*100, 2), "%\n")

# Optional: Confusion matrix for detailed performance
confusion_matrix <- table(Predicted=y_pred, Actual=y_test)
print(confusion_matrix)

The most straightforward way to adapt your binary classifier to multi-class problems is the One-vs-Rest (OvR) approach. Here's how it works:

• For each class, treat it as the "positive" class (label=1) and all others as "negative" (label=-1)
• Train a separate binary classifier for each class
• For test samples, predict the class with the highest f(x) score (not just the sign)

Step-by-Step Implementation #

Assume your y_train now has multi-class labels (e.g., 1, 2, 3 instead of -1,1).

a. Train multiple binary classifiers

# Get unique classes from training labels
unique_classes <- unique(y_train)
num_classes <- length(unique_classes)

# Store coefficients for each classifier
c_list <- list()

# Train one model per class
for (k in 1:num_classes) {
  # Create binary labels: current class = 1, others = -1
  y_binary <- ifelse(y_train == unique_classes[k], 1, -1)
  
  # Compute training kernel matrix (same as your original code)
  K <- gausskernel(X=x_train, sigma=1)
  
  # Solve for coefficients (keep your regularization term)
  reg_term <- nrow(x_train) * diag(nrow(K))
  c_k <- solve(reg_term + K, y_binary)
  
  # Store the coefficients
  c_list[[k]] <- c_k
}

b. Predict on test set

# Compute test-train kernel matrix (same as before)
K_test <- gausskernel_two_sets(X1=x_test, X2=x_train, sigma=1)

# Calculate f(x) scores for each class
f_scores <- matrix(0, nrow=nrow(x_test), ncol=num_classes)
for (k in 1:num_classes) {
  f_scores[,k] <- K_test %*% c_list[[k]]
}

# Predict the class with the highest score
y_pred_multi <- unique_classes[apply(f_scores, 1, which.max)]

c. Evaluate multi-class accuracy

# Multi-class accuracy percentage
multi_accuracy <- mean(y_pred_multi == y_test)
cat("Multi-Class Test Accuracy:", round(multi_accuracy*100, 2), "%\n")

# Optional: Confusion matrix for multi-class performance
multi_confusion <- table(Predicted=y_pred_multi, Actual=y_test)
print(multi_confusion)

Quick Tips #

• Regularization: Your original code uses n*I as the regularization term. You might want to tune this value (use a smaller lambda) via cross-validation to avoid overfitting.
• Speed: For large datasets, nested loops in R can be slow. Consider using vectorized operations or packages like kernlab for optimized kernel computations.

内容的提问来源于stack exchange，提问作者J_Paul_