Optuna调优CatBoostRegressor超参数报无法修改已拟合模型参数错误

• 你将CatBoost模型实例和Pipeline定义在了全局作用域，第一轮试验调用pipe.fit()完成后，绑定在Pipeline里的cbr_model就变为已拟合状态
• CatBoost本身的设计对已拟合模型做了强限制，禁止通过set_params修改已训练完成的模型参数，避免破坏已有训练结果
• XGBoost、LightGBM的Sklearn接口没有设置这类强校验，允许覆盖已拟合模型的参数，所以相同写法可以正常运行

将Pipeline、模型实例的定义移入objective函数内部，每次试验都初始化全新的未拟合模型，不会复用之前已训练的模型实例即可解决问题。

修改后的代码示例：

import numpy as np
import optuna
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.metrics import mean_squared_error
from catboost import CatBoostRegressor

# 特征分类、预处理逻辑可以留在全局，这部分不会被训练修改
cat_cols = [cname for cname in train_data1.columns if train_data1[cname].dtype == 'object']
num_cols = [cname for cname in train_data1.columns if train_data1[cname].dtype in ['int64', 'float64']]

num_trans = Pipeline(steps = [('impute', SimpleImputer(strategy = 'mean')),('scale', StandardScaler())])
cat_trans = Pipeline(steps = [('impute', SimpleImputer(strategy = 'most_frequent')), ('encode', OneHotEncoder(handle_unknown = 'ignore'))])
preproc = ColumnTransformer(transformers = [('cat', cat_trans, cat_cols), ('num', num_trans, num_cols)])

def objective(trial):
    model__depth = trial.suggest_int('model__depth', 2, 10)
    model__iterations = trial.suggest_int('model__iterations', 100, 2000)
    model__subsample = trial.suggest_float('model__subsample', 0.0, 1.0)
    model__learning_rate = trial.suggest_float('model__learning_rate', 0.001, 0.3, log = True)

    # 每次trial都新建全新的模型和Pipeline实例
    cbr_model = CatBoostRegressor(
        random_state = 69, 
        loss_function='RMSE', 
        eval_metric='RMSE', 
        leaf_estimation_method ='Newton', 
        bootstrap_type='Bernoulli', 
        task_type = 'GPU',
        depth=model__depth,
        iterations=model__iterations,
        subsample=model__subsample,
        learning_rate=model__learning_rate
    )
    pipe = Pipeline(steps = [('preproc', preproc), ('model', cbr_model)])
    
    pipe.fit(train_x, train_y)
    pred = pipe.predict(test_x)
    return np.sqrt(mean_squared_error(test_y, pred))

cbr_study = optuna.create_study(direction = 'minimize')
cbr_study.optimize(objective, n_trials = 10)

内容的提问来源于stack exchange，提问作者spectre