修复sklearn Pipeline无best_estimator_报错 正确实现嵌套交叉验证

AttributeError: Pipeline object has no attribute 'best_estimator_'报错来自四个核心问题：

• 属性访问层级错误：你将GridSearchCV作为Pipeline的最后一个步骤，fit后得到的result是Pipeline实例，本身不携带best_estimator_、best_score_、best_params_这类GridSearchCV专属属性，这些属性属于Pipeline中名为clf_cv的GridSearchCV步骤
• 超参数命名不规范：传入GridSearchCV的参数字典没有添加Pipeline对应的步骤名前缀，即使解决属性访问问题，后续超参数搜索也会触发参数不匹配错误
• 逻辑存在数据泄露风险：当前写法先独立执行RFECV做特征选择，再将筛选后的特征输入GridSearchCV做超参搜索，RFECV特征选择过程会用到外层拆分的测试折数据，完全违背嵌套交叉验证防泄露的设计初衷
• 代码遗漏了accuracy变量的计算逻辑，即使修复前面的错误，打印语句也会直接抛出变量未定义的错误

要实现无数据泄露的嵌套交叉验证+RFECV特征选择+GridSearchCV超参优化，需要将RFECV和分类器共同封装为Pipeline，作为GridSearchCV的搜索对象，保证内层交叉验证的每一次拆分，都仅在当前训练折上完成特征选择、超参搜索全流程，全程不触碰验证折/测试折数据。

修正后的可运行代码如下：

from sklearn.model_selection import GridSearchCV, KFold
from sklearn.feature_selection import RFECV
from sklearn.pipeline import Pipeline
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.datasets import make_classification
import numpy as np

# 生成模拟数据集
full_X_train, full_y_train = make_classification(
    n_samples=500, n_features=20, random_state=1, n_informative=10, n_redundant=10
)

def run_model_with_grid_search(
    param_grid={},
    model_name=RandomForestClassifier(random_state=1),
    X_train=full_X_train,
    y_train=full_y_train,
    n_splits_outer=5,
    n_splits_inner=3,
    n_splits_rfecv=5
):
    # 外层交叉验证：用于评估模型真实泛化性能
    cv_outer = KFold(n_splits=n_splits_outer, shuffle=True, random_state=1)
    outer_acc_scores = []
    outer_auc_scores = []
    best_params_list = []

    for train_ix, test_ix in cv_outer.split(X_train):
        split_x_train, split_x_test = X_train[train_ix, :], X_train[test_ix, :]
        split_y_train, split_y_test = y_train[train_ix], y_train[test_ix]

        # 内层交叉验证：用于超参数搜索+特征选择，全程不触碰外层测试折
        cv_inner = KFold(n_splits=n_splits_inner, shuffle=True, random_state=1)
        
        # 把特征选择和模型打包为Pipeline，作为GridSearchCV的搜索对象
        inner_pipeline = Pipeline([
            ('feature_sele', RFECV(estimator=model_name, step=1, cv=n_splits_rfecv, scoring='roc_auc')),
            ('clf', model_name)
        ])

        search = GridSearchCV(
            estimator=inner_pipeline,
            param_grid=param_grid,
            scoring='roc_auc',
            cv=cv_inner,
            refit=True,
            n_jobs=-1
        )

        # 在内层训练折上完成全流程搜索
        search_result = search.fit(split_x_train, split_y_train)
        # 在外层从未参与训练的测试折上做泛化评估
        best_model = search_result.best_estimator_
        y_pred = best_model.predict(split_x_test)
        y_pred_proba = best_model.predict_proba(split_x_test)[:,1]

        # 计算单折评估指标
        fold_acc = accuracy_score(split_y_test, y_pred)
        fold_auc = roc_auc_score(split_y_test, y_pred_proba)
        outer_acc_scores.append(fold_acc)
        outer_auc_scores.append(fold_auc)
        best_params_list.append(search_result.best_params_)

        print(f'>fold acc={fold_acc:.3f}, inner best val auc={search_result.best_score_:.3f}, best cfg={search_result.best_params_}')

    # 输出外层交叉验证的整体泛化性能
    print('='*60)
    print(f'Outer CV average accuracy: {np.mean(outer_acc_scores):.3f} ± {np.std(outer_acc_scores):.3f}')
    print(f'Outer CV average AUC: {np.mean(outer_auc_scores):.3f} ± {np.std(outer_auc_scores):.3f}')

    return search_result, outer_acc_scores, outer_auc_scores, best_params_list

# 注意：待搜索参数名必须加`clf__`前缀，对应Pipeline中clf步骤的参数
param_grid = {
    'clf__min_samples_leaf': [1,3,5],
    'clf__n_estimators': [50, 100]
}

run_model_with_grid_search(param_grid=param_grid)

• 调整Pipeline和GridSearchCV的嵌套关系：将包含特征选择、分类器的Pipeline作为GridSearchCV的estimator，而非将GridSearchCV作为Pipeline的步骤，从结构上保证特征选择、超参搜索都在内层CV的训练折上完成，彻底避免数据泄露
• 规范超参数命名：所有待搜索的分类器参数前添加clf__前缀（对应内层Pipeline中clf命名的分类器步骤），保证GridSearchCV可以正确识别待调优参数
• 修正属性访问逻辑：fit后得到的结果是GridSearchCV实例，可直接访问best_estimator_等属性，无需跨Pipeline层级查找
• 补全评估指标计算逻辑，新增外层交叉验证的性能均值、标准差汇总输出，得到更可靠的模型泛化性能估计
• 为所有涉及随机过程的组件添加固定随机种子，保证运行结果可复现

内容的提问来源于stack exchange，提问作者Slowat_Kela