如何用Random Forest填补数据集空缺并生成完整数据表？

核心需求与疑问 #

• 已基于划分好的训练/测试集训练Random Forest回归模型，需将其应用到存在Fch4缺失的目标数据集，生成包含填补后数值的完整数据表
• 疑问：当待填补数据中，除目标变量Fch4外，特征列Ta、Fe也存在缺失时，是否会影响填补过程

训练数据与模型代码 #

# 训练数据集构造
TIMESTAMP <- c("2019-05-31 17:00:00", "2019-05-31 17:30:00", "2019-05-31 18:00:00", 
               "2019-05-31 18:30:00", "2019-05-31 19:00:00", "2019-05-31 19:30:00", 
               "2019-05-31 20:00:00", "2019-05-31 20:30:00", "2019-05-31 21:00:00", 
               "2019-05-31 21:30:00")
RH<-c(38, 40, 41, 42, 44, 49, 65, 72, 74, 77)
Fch4 <- c(0.045, -0.002, 0.001, 0.004, 0, -0.013, 0.004,-0.003, -0.001,-0.002)
distance <- c(1000,1000,180,125.35,1000,180,1000,5.50,180,1000)
Ta <-c(29.52, 29.01, 29.04, 28.39, 27.87, 26.68, 23.28, 21.16, 19.95, 19.01)
Fe<- c(95.16, 68.95, 68.62, 39.24, 35.04, 27.26, -2.60, 5.09,7.28, 2.08)

dd_train <- data.frame(TIMESTAMP, RH, Fch4, distance, Ta, Fe)

# 划分训练集与测试集并训练RF模型
library(caret)
library(randomForest)
set.seed(1)
inTraining <- createDataPartition(dd_train$Fch4, p = 0.65, list=FALSE)
training <- dd_train[inTraining,]
testing <- dd_train[-inTraining,]

set.seed(1)
pfpfit <- randomForest(Fch4 ~ ., data = training, ntree=500)
predicted <- predict(pfpfit, newdata = testing)

待填补的目标数据集 #

# 待填补数据集构造
TIMESTAMP <- c("2019-05-31 17:00:00", "2019-05-31 17:30:00", "2019-05-31 18:00:00", 
               "2019-05-31 18:30:00", "2019-05-31 19:00:00", "2019-05-31 19:30:00", 
               "2019-05-31 20:00:00", "2019-05-31 20:30:00", "2019-05-31 21:00:00", 
               "2019-05-31 21:30:00")
RH<-c(38, 40, 41, 42, 44, 49, 65, 72, 74, 77)
Fch4 <- c(NA, -0.002, 0.001, 0.004, NA, -0.013, 0.004,NA, -0.001,-0.002)
distance <- c(1000,1000,180,125.35,1000,180,1000,5.50,180,1000)
Ta <-c(29.52, 29.01, NA, 28.39, 27.87, 26.68, 23.28, NA, 19.95, 19.01)
Fe<- c(NA, NA, 68.62, 39.24, 35.04, 27.26, -2.60, NA,7.28, 2.08)

dd_missing <- data.frame(TIMESTAMP, RH, Fch4, distance, Ta, Fe)

解决方案 #

1. 特征缺失的影响说明 #

Random Forest的predict函数无法直接处理新数据中特征列的缺失值——模型训练时依赖完整的特征输入，若新数据的Ta/Fe存在NA，对应行将无法生成Fch4的预测结果。因此必须先填补特征列的缺失，再处理Fch4的缺失。

2. 完整填补步骤 #

步骤1：填补特征列（Ta、Fe）的缺失

利用训练数据中的完整样本，分别训练针对Ta和Fe的Random Forest回归模型，填补待填补数据中的对应缺失值：

# 填补Ta的缺失
set.seed(1)
ta_model <- randomForest(Ta ~ RH + distance + Fe + Fch4, data = dd_train[!is.na(dd_train$Ta),])
dd_missing$Ta[is.na(dd_missing$Ta)] <- predict(ta_model, newdata = dd_missing[is.na(dd_missing$Ta),])

# 填补Fe的缺失
set.seed(1)
fe_model <- randomForest(Fe ~ RH + distance + Ta + Fch4, data = dd_train[!is.na(dd_train$Fe),])
dd_missing$Fe[is.na(dd_missing$Fe)] <- predict(fe_model, newdata = dd_missing[is.na(dd_missing$Fe),])

步骤2：填补目标变量Fch4的缺失

用之前训练好的pfpfit模型，对已填补完特征的数据集生成Fch4预测值，替换原有的NA：

# 填补Fch4的缺失
dd_missing$Fch4[is.na(dd_missing$Fch4)] <- predict(pfpfit, newdata = dd_missing[is.na(dd_missing$Fch4),])

3. 最终填补后数据集 #

# 查看完整数据集
print(dd_missing)
# 输出结果与示例一致：
#             TIMESTAMP RH   Fch4 distance    Ta    Fe
# 1  2019-05-31 17:00:00 38  0.045   1000.00 29.52 95.16
# 2  2019-05-31 17:30:00 40 -0.002   1000.00 29.01 68.95
# 3  2019-05-31 18:00:00 41  0.001    180.00 29.04 68.62
# 4  2019-05-31 18:30:00 42  0.004    125.35 28.39 39.24
# 5  2019-05-31 19:00:00 44  0.000   1000.00 27.87 35.04
# 6  2019-05-31 19:30:00 49 -0.013    180.00 26.68 27.26
# 7  2019-05-31 20:00:00 65  0.004   1000.00 23.28 -2.60
# 8  2019-05-31 20:30:00 72 -0.003      5.50 21.16  5.09
# 9  2019-05-31 21:00:00 74 -0.001    180.00 19.95  7.28
# 10 2019-05-31 21:30:00 77 -0.002   1000.00 19.01  2.08

内容的提问来源于stack exchange，提问作者shrimp