Hey there! Let's walk through this step by step—first verifying that your nested spatial resampling setup is correct, then figuring out how to get feature importance from your tuned SVM model (since SVMs don't have built-in importance scores like tree-based models do).

First, let's make sure your implementation of outer SpRepCV and inner SpCV is aligned with best practices for spatial modeling. Here's a complete, reproducible example using mlr and e1071::svm:

Step 1: Load Required Packages & Prepare Spatial Data #

library(mlr)
library(e1071)
library(sp)

# Assume you have a data frame with a target variable and spatial coordinates (x, y)
# Convert to a SpatialPointsDataFrame (required for mlr's spatial resampling)
spatial_data <- SpatialPointsDataFrame(
  coords = your_data[, c("x", "y")],
  data = your_data[, !names(your_data) %in% c("x", "y")]
)

# Create a task (use makeClassifTask for classification)
task <- makeRegrTask(data = spatial_data, target = "your_target_variable")

Step 2: Define Resampling Strategies #

• Outer resampling: Spatial Repeated Cross-Validation (SpRepCV) to get unbiased performance estimates
• Inner resampling: Spatial Cross-Validation (SpCV) for hyperparameter tuning

# Outer: 3 repetitions of 5-fold spatial CV
outer_resampling <- makeResampleDesc("SpRepCV", folds = 5, reps = 3, spatial = TRUE)

# Inner: 5-fold spatial CV for tuning
inner_resampling <- makeResampleDesc("SpCV", folds = 5, spatial = TRUE)

Step 3: Define Tuning Parameters & Tuner #

SVMs rely heavily on gamma (kernel width) and cost (penalty term)—don't skip tuning cost! We'll use random search for efficiency:

# Define parameter search space
param_set <- makeParamSet(
  makeNumericParam("gamma", lower = 1e-4, upper = 10),
  makeNumericParam("cost", lower = 0.1, upper = 100)
)

# Use random search (faster than grid search for high-dimensional spaces)
tuner <- makeTuner("randomSearch", maxit = 50)

Step 4: Wrap Learner & Run Nested Resampling #

We'll use makeTuneWrapper to combine the learner with tuning, and save the tuned models from each outer fold with extract:

# Create base SVM learner (use classif.svm for classification)
svm_learner <- makeLearner("regr.svm", predict.type = "response")

# Wrap learner with tuning logic
tuned_svm <- makeTuneWrapper(
  learner = svm_learner,
  resampling = inner_resampling,
  par.set = param_set,
  tuner = tuner,
  show.info = TRUE
)

# Run nested resampling and extract tuned models
nested_results <- resample(
  learner = tuned_svm,
  task = task,
  resampling = outer_resampling,
  extract = function(x) x$learner.model, # Save each tuned model
  show.info = TRUE
)

This setup is correct for unbiased spatial model tuning—you're avoiding data leakage by keeping tuning within each outer fold's training data.

Unlike tree-based models, e1071::svm doesn't have built-in feature importance scores. The standard workaround here is permutation importance: we randomly shuffle each feature's values and measure how much model performance drops. A larger drop means the feature is more important.

Step 1: Extract Tuned Models from Nested Results #

First, pull out all the tuned models from each outer fold:

tuned_models <- nested_results$extract

Step 2: Define a Function to Calculate Permutation Importance #

We'll use mlr's generateFeatureImportanceData to handle the permutation logic:

calculate_perm_importance <- function(model, task, n_permutations = 5) {
  # Create a learner with the tuned hyperparameters
  tuned_learner <- makeLearner(
    class(model),
    par.vals = model$par.vals,
    predict.type = ifelse(task$task.type == "classif", "prob", "response")
  )
  
  # Choose an appropriate performance measure (adjust based on task type)
  performance_measure <- ifelse(task$task.type == "classif", auc, mse)
  
  # Generate importance data
  importance_data <- generateFeatureImportanceData(
    task = task,
    learner = tuned_learner,
    measure = performance_measure,
    nperm = n_permutations,
    sample.size = nrow(task$data)
  )
  
  return(importance_data$res)
}

Step 3: Compute & Aggregate Importance Across All Models #

Since we have multiple tuned models (one per outer fold), we'll calculate importance for each and take the average:

# Calculate importance for each tuned model
all_importance <- lapply(tuned_models, calculate_perm_importance, task = task)

# Combine results into a single data frame
importance_df <- do.call(rbind, all_importance)

# Calculate mean importance per feature
mean_feature_importance <- aggregate(
  importance_df$importance,
  by = list(feature = importance_df$feature),
  FUN = mean
)
colnames(mean_feature_importance) <- c("feature", "mean_importance")

# Sort features by importance (descending)
mean_feature_importance <- mean_feature_importance[order(-mean_feature_importance$mean_importance), ]

Step 4: Visualize the Results #

barplot(
  mean_feature_importance$mean_importance,
  names.arg = mean_feature_importance$feature,
  las = 2, # Rotate feature names for readability
  main = "SVM Feature Importance (Permutation-Based)",
  ylab = "Mean Performance Drop"
)

• Spatial Resampling Checks: Ensure your SpatialPointsDataFrame has valid coordinates—mlr's spatial resampling uses these to create folds that avoid spatial autocorruption.
• Tuning Efficiency: If you have a large dataset, reduce maxit in the tuner or switch to grid search with a smaller parameter space.
• Permutation Stability: Increase n_permutations if you want more stable importance scores (this will increase computation time).

内容的提问来源于stack exchange，提问作者raff-k