如何在Optuna调优LightGBM时返回平均准确率而非对数损失

问题说明 #

我正在用Optuna调优LightGBM多分类模型的超参数，希望把平均准确率作为试验的返回指标，替代当前使用的对数损失。目前已经能打印每轮交叉验证的准确率，但不知道如何将这些准确率取平均后作为Optuna的优化目标返回。

修改后的完整代码 #

import numpy as np
import optuna
from optuna.integration.lightgbm import LightGBMPruningCallback
from sklearn.model_selection import StratifiedKFold
from sklearn.metrics import accuracy_score
import lightgbm as lg

def objective(trial, X, y):
    param_grid = {
        # "device_type": trial.suggest_categorical("device_type", ['gpu']),
        "n_estimators": trial.suggest_categorical("n_estimators", [10000]),
        "learning_rate": trial.suggest_float("learning_rate", 0.01, 0.05),
        "num_leaves": trial.suggest_int("num_leaves", 2, 50, step=2),
        "max_depth": trial.suggest_int("max_depth", 1, 5),
        "min_data_in_leaf": trial.suggest_int("min_data_in_leaf", 5, 100, step=5),
        "lambda_l1": trial.suggest_int("lambda_l1", 0, 100, step=5),
        "lambda_l2": trial.suggest_int("lambda_l2", 0, 100, step=5),
        "min_gain_to_split": trial.suggest_float("min_gain_to_split", 0, 15),
        "bagging_fraction": trial.suggest_float(
            "bagging_fraction", 0.2, 0.90, step=0.1
        ),
        "bagging_freq": trial.suggest_categorical("bagging_freq", [1]),
        "feature_fraction": trial.suggest_float(
            "feature_fraction", 0.2, 0.90, step=0.1
        ),
    }

    cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=1121218)
    # 改为存储准确率的数组
    cv_scores = np.empty(5)

    for idx, (train_idx, test_idx) in enumerate(cv.split(X, y)):
        X_train, X_test = X[train_idx], X[test_idx]
        y_train, y_test = y[train_idx], y[test_idx]

        model = lg.LGBMClassifier(objective="multiclass", num_classes=3, **param_grid)
        model.fit(
            X_train,
            y_train,
            eval_set=[(X_test, y_test)],
            # 把评估指标改成多分类错误率（与准确率反向）
            eval_metric="multi_error",
            early_stopping_rounds=100,
            # 回调函数同步监控对应指标
            callbacks=[
                LightGBMPruningCallback(trial, "multi_error")
            ],
        )
        # 计算当前折的准确率
        acc = accuracy_score(y_test, model.predict(X_test))
        cv_scores[idx] = acc
        print(f"Testing accuracy {acc:.4f}")

    # 返回平均准确率作为优化目标
    return np.mean(cv_scores)

# 准确率越大越好，优化方向改为最大化
study = optuna.create_study(direction="maximize", study_name="LGBM Classifier")
func = lambda trial: objective(trial, X_train, y_train)
study.optimize(func, n_trials=20)

关键修改点 #

• 替换指标存储逻辑：将cv_scores从存储对数损失改为存储每折的准确率
• 调整训练评估指标：把eval_metric从multi_logloss改为multi_error（LightGBM无直接多分类准确率指标，multi_error为错误率，与准确率反向适配剪枝回调）
• 同步剪枝回调指标：LightGBMPruningCallback的监控指标改为multi_error，保证剪枝逻辑与当前评估标准一致
• 计算并保存准确率：用accuracy_score计算单折准确率，替代原有的对数损失计算
• 修改优化方向：创建Study时将direction从minimize改为maximize，匹配准确率越高越好的优化需求

内容的提问来源于stack exchange，提问作者Youngun