如何合并多个GridSearchCV实例以实现参数空间并行探索

当需要突破n_jobs限制，并行探索更大的参数空间时，可以将GridSearchCV的参数集拆分为多个子集，用独立任务分别执行搜索。之后无需手动遍历cv_results_对比结果，可通过以下方式合并多个GridSearchCV实例，直接获取全局最优参数。

原参数集与拆分示例 #

原完整参数集 #

all_params = {'n_estimators': [10, 20], 'min_samples_leaf': [10, 20]}

拆分后独立执行代码 #

将参数集拆分为两个子集，分别在独立脚本中运行：

from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier  # 示例评估器
import joblib

# Job 1：独立脚本运行
params1 = {'n_estimators': [10], 'min_samples_leaf': [10, 20]}
grid_search1 = GridSearchCV(
    estimator=RandomForestClassifier(),
    param_grid=params1,
    cv=5,
    scoring='accuracy'
)
grid_search1.fit(X_train, y_train)
joblib.dump(grid_search1, 'grid_search1.joblib')  # 保存实例

# Job 2：独立脚本运行
params2 = {'n_estimators': [20], 'min_samples_leaf': [10, 20]}
grid_search2 = GridSearchCV(
    estimator=RandomForestClassifier(),
    param_grid=params2,
    cv=5,
    scoring='accuracy'
)
grid_search2.fit(X_train, y_train)
joblib.dump(grid_search2, 'grid_search2.joblib')  # 保存实例

合并GridSearchCV实例的实现 #

由于scikit-learn原生不支持直接用+合并GridSearchCV实例，我们可以通过合并cv_results_并重新计算最优参数来实现：

加载保存的实例 #

import joblib

grid_search1 = joblib.load('grid_search1.joblib')
grid_search2 = joblib.load('grid_search2.joblib')

合并函数实现 #

def merge_grid_searches(grid_list):
    # 校验所有实例的核心配置一致
    assert all(grid.estimator.__class__ == grid_list[0].estimator.__class__ for grid in grid_list), "所有实例必须使用相同评估器"
    assert all(grid.cv == grid_list[0].cv for grid in grid_list), "所有实例必须使用相同交叉验证策略"
    assert all(grid.scoring == grid_list[0].scoring for grid in grid_list), "所有实例必须使用相同评分指标"

    # 初始化合并后的cv_results_
    merged_results = grid_list[0].cv_results_.copy()
    # 遍历合并剩余实例的结果
    for grid in grid_list[1:]:
        for key in merged_results:
            # 拼接列表/数组类型的结果
            if isinstance(merged_results[key], list):
                merged_results[key].extend(grid.cv_results_[key])
            else:
                merged_results[key] = merged_results[key].tolist() + grid.cv_results_[key].tolist()

    # 创建空的GridSearchCV对象承载合并结果
    merged_grid = GridSearchCV(estimator=grid_list[0].estimator, param_grid={})
    merged_grid.cv_results_ = merged_results
    merged_grid.scoring = grid_list[0].scoring
    merged_grid.refit = grid_list[0].refit

    # 计算最优参数
    score_key = 'mean_test_score'
    if merged_grid.refit and isinstance(merged_grid.scoring, dict):
        # 多指标场景，使用refit指定的指标
        score_key = f'mean_test_{merged_grid.refit}'
    
    # 判断评分是极大化还是极小化
    from sklearn.metrics._scorer import _check_multimetric_scoring
    scorers = _check_multimetric_scoring(merged_grid.estimator, scoring=merged_grid.scoring)
    scorer = scorers[merged_grid.refit] if isinstance(merged_grid.scoring, dict) else list(scorers.values())[0]
    
    if scorer._sign > 0:
        best_idx = merged_results[score_key].index(max(merged_results[score_key]))
    else:
        best_idx = merged_results[score_key].index(min(merged_results[score_key]))

    # 设置最佳属性
    merged_grid.best_index_ = best_idx
    merged_grid.best_params_ = merged_results['params'][best_idx]
    merged_grid.best_score_ = merged_results[score_key][best_idx]

    return merged_grid

使用合并函数获取最优参数 #

full_grid = merge_grid_searches([grid_search1, grid_search2])
print(full_grid.best_params_)

注意事项 #

• 所有拆分的GridSearchCV实例必须使用相同的评估器、交叉验证策略、评分指标，否则合并结果无效
• 多指标评分场景下，需确保所有实例的refit参数一致，合并时会自动使用该指标筛选最优参数
• 推荐使用joblib保存/加载GridSearchCV实例，相比pickle更适合处理包含numpy数组的scikit-learn对象

内容的提问来源于stack exchange，提问作者BML