如何在R语言中处理缺失值(NA)？含训练集与测试集处理方案

Hey there! Handling missing values (NA) in training and test sets is a super common task in R, and it’s critical to do it correctly to avoid data leakage (using test set data to inform your training process). Let’s break down the most practical methods step by step:

This is the simplest approach, great for quick fixes with low-missing-rate features. The golden rule here: calculate the statistic only from the training set, then apply it to both training and test sets.

Example for continuous variables (mean/median): #

Suppose we have train_df and test_df with a continuous column age that has NAs:

# Calculate median from training set (ignoring NAs)
median_age <- median(train_df$age, na.rm = TRUE)

# Fill NAs in training set
train_df$age <- ifelse(is.na(train_df$age), median_age, train_df$age)

# Use the SAME median to fill test set NAs
test_df$age <- ifelse(is.na(test_df$age), median_age, test_df$age)

(Use mean() instead of median() if your data is normally distributed; median is better for skewed data.)

Example for categorical variables (mode): #

For categorical columns like gender, we use the mode (most frequent value). First, write a quick mode function:

get_mode <- function(x) {
  unique_vals <- unique(x[!is.na(x)])
  unique_vals[which.max(tabulate(match(x, unique_vals)))]
}

# Get mode from training set
mode_gender <- get_mode(train_df$gender)

# Fill NAs in both sets
train_df$gender <- ifelse(is.na(train_df$gender), mode_gender, train_df$gender)
test_df$gender <- ifelse(is.na(test_df$gender), mode_gender, test_df$gender)

If you have multiple columns to impute, the caret package makes this efficient and avoids repetition. It automatically computes imputation parameters from the training set and applies them to both sets.

library(caret)

# Define preprocessing: choose method(s) - "meanImpute", "medianImpute", or "knnImpute"
pre_process <- preProcess(train_df, method = "medianImpute")

# Apply to training and test sets
train_imputed <- predict(pre_process, train_df)
test_imputed <- predict(pre_process, test_df)

• knnImpute uses k-nearest neighbors to predict missing values (great for more complex relationships) — it automatically standardizes data too.
• You can combine methods, e.g., method = c("medianImpute", "knnImpute") for different columns.

For datasets with complex relationships, use a model to predict missing values. The mice package is popular for this (it supports multiple imputation, but we’ll show single imputation here):

library(mice)

# Train imputation model on training set (using random forest: method = "rf")
imputation_model <- mice(train_df, method = "rf", m = 1, printFlag = FALSE)

# Get imputed training set
train_imputed <- complete(imputation_model)

# For test set: Avoid using the mice model directly (risk of leakage). Instead,
# you can train a separate model using the imputed training data to predict test set NAs,
# or use a simpler method if the missing rate is low.

Key Rules to Remember: #

• No data leakage ever: Never use test set values to calculate imputation stats or train imputation models. All parameters must come from the training set.
• Match imputation method to variable type: Use mean/median for continuous, mode/model for categorical.
• If a feature has >30% NAs, consider dropping it instead of imputing — imputation might introduce too much noise.

内容的提问来源于stack exchange，提问作者ph9675