如何在R语言H2O GBM中实现自定义stopping_metric？

Got it, let's fix this issue for you! The error you're seeing is straightforward—when you set stopping_metric='custom', H2O needs you to provide the actual custom metric function it should use to evaluate early stopping. Here's a step-by-step implementation to make this work:

Step 1: Define Your Custom Metric Function #

First, create a function that accepts two H2OFrames as inputs:

• The first frame holds the actual response values from your dataset
• The second frame holds the predicted values (for classification tasks, this will include predicted probabilities or class labels, depending on your needs)

The function must return a single numeric value—this is the score H2O will use to decide when to stop training. For example, let's build a custom F1-score metric for your binary classification task:

custom_f1_metric <- function(actual, predicted) {
  # Convert H2OFrames to R data frames for calculation
  actual_vals <- as.data.frame(actual)$y
  predicted_probs <- as.data.frame(predicted)$p1
  
  # Convert probabilities to class labels (0/1 threshold at 0.5)
  predicted_vals <- ifelse(predicted_probs > 0.5, 1, 0)
  
  # Calculate F1 score components
  true_positive <- sum(actual_vals == 1 & predicted_vals == 1)
  false_positive <- sum(actual_vals == 0 & predicted_vals == 1)
  false_negative <- sum(actual_vals == 1 & predicted_vals == 0)
  
  precision <- true_positive / (true_positive + false_positive)
  recall <- true_positive / (true_positive + false_negative)
  f1_score <- 2 * (precision * recall) / (precision + recall)
  
  # Return the metric value (H2O maximizes by default; adjust if you need minimization)
  return(f1_score)
}

Step 2: Train the GBM with the Custom Metric #

Update your h2o.gbm() call to include the custom_metric parameter pointing to your function, alongside stopping_metric='custom'. Here's your modified working code:

library(h2o)
h2o.init()

# Generate sample training data
x <- data.frame( 
  x = rnorm(1000), 
  z = rnorm(1000), 
  y = factor(sample(0:1, 1000, replace = TRUE)) 
)
train <- as.h2o(x)

# Train GBM with custom stopping metric
model <- h2o.gbm(
  x = c('x','z'), 
  y = 'y', 
  training_frame = train, 
  stopping_metric = 'custom', 
  stopping_rounds = 3,
  custom_metric = custom_f1_metric,  # This is the missing piece!
  score_tree_interval = 1,  # Ensure metric is calculated every tree round
  seed = 123  # For reproducible results
)

Key Tips to Remember: #

• Metric Direction: H2O defaults to maximizing the custom metric. If your metric should be minimized (like mean squared error), adjust your function to return a negative value or set stopping_tolerance to a small positive number.
• Cluster Efficiency: For large datasets, avoid converting H2OFrames to R data frames—use H2O's built-in functions (like h2o.sum(), h2o.mean()) to keep calculations in the H2O cluster.
• Version Check: This functionality is only available in H2O v3.22.1.1+, so your upgrade from v3.10.0.9 was necessary.

内容的提问来源于stack exchange，提问作者davide