咨询：如何用R语言e1071包实现SVM贪心维度降维

Got it, let's break down how to implement this greedy feature selection with SVM using R's e1071 package—perfect for your fungal classification task where you want to add compounds (features) one by one to build the optimal model. The core idea is to start with no features, then iteratively add the single feature that gives the biggest performance boost, stopping when adding more features doesn't help.

Step 1: Setup & Data Prep #

First, make sure you have the necessary packages loaded, and confirm your training/test sets are structured correctly (your fungal class should be the response variable y, with all other columns being compound features).

# Load required packages
library(e1071)   # For SVM implementation
library(caret)   # For performance evaluation (confusion matrix)

Step 2: Greedy Selection Implementation #

Here's a complete, commented script to run the greedy feature selection:

# Initialize tracking variables
selected_features <- c()
remaining_features <- setdiff(colnames(train_data)[colnames(train_data) != "y"], selected_features)
best_overall_acc <- 0
best_feature_set <- c()
performance_log <- data.frame(step = integer(), num_features = integer(), test_accuracy = numeric())

# Start the greedy selection loop
while (length(remaining_features) > 0) {
  temp_performance <- c()
  
  # Test each remaining feature one by one
  for (feat in remaining_features) {
    # Build a temporary feature set with current selected + this new feature
    temp_feats <- c(selected_features, feat)
    
    # Train linear SVM (adjust kernel/params to match your paper's setup)
    svm_model <- svm(
      formula = y ~ .,
      data = train_data[, c("y", temp_feats)],
      kernel = "linear",  # Common choice for feature selection; tweak if needed
      cost = 1            # Adjust regularization parameter as needed
    )
    
    # Predict on test set and calculate accuracy
    svm_pred <- predict(svm_model, newdata = test_data[, temp_feats])
    acc <- confusionMatrix(svm_pred, test_data$y)$overall["Accuracy"]
    temp_performance <- c(temp_performance, acc)
  }
  
  # Find the feature that gave the best performance
  top_feat_idx <- which.max(temp_performance)
  top_feat <- remaining_features[top_feat_idx]
  top_acc <- temp_performance[top_feat_idx]
  
  # Update if this improves our best overall performance
  if (top_acc > best_overall_acc) {
    selected_features <- c(selected_features, top_feat)
    best_overall_acc <- top_acc
    best_feature_set <- selected_features
    
    # Log progress
    performance_log <- rbind(
      performance_log,
      data.frame(
        step = nrow(performance_log) + 1,
        num_features = length(selected_features),
        test_accuracy = round(top_acc, 4)
      )
    )
    
    # Remove the selected feature from the remaining pool
    remaining_features <- setdiff(remaining_features, top_feat)
    cat(paste0("Added feature: ", top_feat, " | Current test accuracy: ", round(top_acc, 4), "\n"))
  } else {
    # No improvement—stop the loop early
    cat("No performance gain with remaining features. Stopping selection.\n")
    break
  }
}

# Final results
cat("\n=== Greedy Selection Results ===\n")
cat("Best feature set: ", paste(best_feature_set, collapse = ", "), "\n")
cat("Highest test accuracy: ", round(best_overall_acc, 4), "\n")

# Optional: Plot performance over feature count
plot(
  performance_log$num_features, performance_log$test_accuracy,
  type = "b", pch = 16, col = "steelblue",
  xlab = "Number of Selected Compounds",
  ylab = "Test Set Accuracy",
  main = "Greedy Feature Selection Performance"
)

Key Notes to Customize #

• Performance Metric: I used accuracy here, but if your fungal classes are imbalanced, swap it for F1-score (confusionMatrix(...)$byClass["F1"]) or another metric that fits your task.
• SVM Parameters: Adjust the kernel (e.g., "radial") or cost (regularization strength) to match the setup in your reference paper.
• Cross-Validation: For more robust evaluation, replace the test set accuracy with cross-validation on the training set (use trainControl from caret to set up CV folds).
• Speed: If you have hundreds of features, this loop can be slow. Consider using parallel processing (e.g., doParallel package) to speed up the per-feature model training.

内容的提问来源于stack exchange，提问作者Yuan.guo