mlr3嵌套重采样中extract_inner_fselect_results返回NULL的解决建议
问题描述
在mlr3的嵌套重采样流程中:
- 若不指定
store_models=TRUE,运行后报错Error: No model stored - 若同时在
AutoFSelector和外层resample中设置store_models=TRUE,会因内存占用过高导致RStudio崩溃 - 仅在
AutoFSelector中设置store_models=TRUE时,能正常提取性能指标,但调用extract_inner_fselect_results()和extract_inner_fselect_archives()返回NULL,无法获取特征选择的相关结果
用户的原始代码如下:
MSvCon<-read.csv("MS v Control Proteomics Final.csv", row.names=1) MSvCon$Status<-as.factor(MSvCon$Status) MSvCon[,2:4399]<-scale(MSvCon[,2:4399], center=TRUE, scale=TRUE) set.seed(123, "L'Ecuyer") task = as_task_classif(MSvCon, target = "Status") learner = lrn("classif.ranger", importance = "impurity", num.trees=10000) set_threads(learner, n = 8) measure = msr("classif.fbeta", beta=1, average="micro") terminator = trm("none") resampling_inner = rsmp("repeated_cv", folds = 10, repeats = 10) at = AutoFSelector$new( learner = learner, resampling = resampling_inner, measure = measure, terminator = terminator, fselect = fs("rfe", n_features = 1, feature_fraction = 0.5, recursive = FALSE), store_models=TRUE) resampling_outer = rsmp("repeated_cv", folds = 10, repeats = 10) rr = resample(task, at, resampling_outer)
解决方案
核心问题是:外层resample未保存AutoFSelector的特征选择结果,同时全量保存模型会导致内存溢出。可通过以下方式平衡需求与内存占用:
- 精准控制
AutoFSelector的保存内容:关闭完整模型保存(store_models = FALSE)避免内存占用过高,同时开启特征选择结果和存档的保存(store_fselect_results = TRUE、store_fselect_archive = TRUE),确保能提取所需的特征信息。 - 外层
resample需保存AutoFSelector实例:设置store_models = TRUE,但由于AutoFSelector已关闭模型保存,此时仅会保存特征选择相关的轻量数据,不会导致内存崩溃。 - 额外内存优化:适当降低
classif.ranger的num.trees参数(比如从10000调整为1000),在性能可接受的前提下进一步减少内存占用;提取完特征选择结果后,立即删除重采样结果对象rr手动释放内存。
代码调整示例
修改后的关键代码部分:
# 初始化AutoFSelector,精准控制保存内容 at = AutoFSelector$new( learner = learner, resampling = resampling_inner, measure = measure, terminator = terminator, fselect = fs("rfe", n_features = 1, feature_fraction = 0.5, recursive = FALSE), store_models = FALSE, # 不保存完整模型,降低内存 store_fselect_results = TRUE, # 保存特征选择结果 store_fselect_archive = TRUE # 保存特征选择存档 ) # 外层resample开启保存,但仅保存轻量的特征选择结果 rr = resample(task, at, resampling_outer, store_models = TRUE) # 现在可以正常提取特征选择结果 inner_results = extract_inner_fselect_results(rr) inner_archives = extract_inner_fselect_archives(rr)
内容的提问来源于stack exchange,提问作者awaldman
相关产品推荐
相关产品推荐

