LightGBM多分类中lgb.cv与cross_val_score结果差异排查

在LightGBM多分类任务中，使用原生API的lgb.cv与Scikit-learn的cross_val_score执行交叉验证时，预期结果相近，但实际二者差异显著。

初始复现代码 #

import lightgbm as lgb
import pandas as pd
from sklearn import datasets
from sklearn.metrics import log_loss
from sklearn.model_selection import cross_val_score

from typing import Any, Dict, List


def log_loss_scorer(clf, X, y):
    y_pred = clf.predict_proba(X)
    return log_loss(y, y_pred)


iris = datasets.load_iris()
features = pd.DataFrame(columns=["f1", "f2", "f3", "f4"], data=iris.data)
target = pd.Series(iris.target, name="target")
# 1) Native API
dataset = lgb.Dataset(features, target, feature_name=list(features.columns), free_raw_data=False)

native_params: Dict[str, Any] = {
    "objective": "multiclass", "boosting_type": "gbdt", "learning_rate": 0.05, "num_class": 3, "seed": 41
}
cv_logloss_native: float = lgb.cv(
    native_params, dataset, num_boost_round=1000, nfold=5, metrics="multi_logloss", seed=41, stratified=False,
    shuffle=False
)['valid multi_logloss-mean'][-1]

# 2) ScikitLearn API
model_scikit = lgb.LGBMClassifier(
    objective="multiclass", boosting_type="gbdt", learning_rate=0.05, n_estimators=1000, random_state=41
)
cv_logloss_scikit_list: List[float] = cross_val_score(
    model_scikit, features, target, scoring=log_loss_scorer
)
cv_logloss_scikit: float = sum(cv_logloss_scikit_list) / len(cv_logloss_scikit_list)
print(f"Native logloss CV {cv_logloss_native}; Scikit logloss CV train {cv_logloss_scikit}")

初始运行结果 #

原生API得到的log_loss为 0.8803800291063604，Scikit-learn API得到的为 0.37528027519836027。更换不同指标后差异仍明显。

后续尝试的调整 #

按照建议禁用原生API的multi_logloss指标，自定义log_loss metric：

def log_loss_custom_metric(y_pred, data: lgb.Dataset):
    y_true = data.get_label()
    loss_value = log_loss(y_true, y_pred)
    return "custom_logloss", loss_value, True

并通过feval参数传入原生API：

cv_logloss_native: float = lgb.cv(native_params, dataset, num_boost_round=1000, nfold=5, feval=log_loss_custom_metric, shuffle=True)["valid custom_logloss-mean"][-1]

但结果仍差异较大：原生API为0.58，Scikit-learn API为0.37。

上述代码基于鸢尾花数据集可100%复现，恳请告知差异根源及对齐结果的方法。

内容的提问来源于stack exchange，提问作者riccio777