使用auto-sklearn训练鸢尾花分类模型时遇Dummy预测崩溃错误

在鸢尾花数据集上使用auto-sklearn训练分类模型，调用fit方法时触发以下错误：

ValueError: (' Dummy prediction failed with run state StatusType.CRASHED and additional output: {'traceback': 'Traceback (most recent call last):\n File "/home/minura/anaconda3/envs/auto-sklearn/lib/python3.10/site-packages/autosklearn/evaluation/init.py", line 55, in fit_predict_try_except_decorator\n return ta(queue=queue, **kwargs)\n File "/home/minura/anaconda3/envs/auto-sklearn/lib/python3.10/site-packages/autosklearn/evaluation/train_evaluator.py", line 1407, in eval_cv\n evaluator.fit_predict_and_loss(iterative=iterative)\n File "/home/minura/anaconda3/envs/auto-sklearn/lib/python3.10/site-packages/autosklearn/evaluation/train_evaluator.py", line 597, in fit_predict_and_loss\n train_loss = {\n File "/home/minura/anaconda3/envs/auto-sklearn/lib/python3.10/site-packages/autosklearn/evaluation/train_evaluator.py", line 599, in \n [train_losses[i][str(metric)] for i in range(self.num_cv_folds)],\n File "/home/minura/anaconda3/envs/auto-sklearn/lib/python3.10/site-packages/autosklearn/evaluation/train_evaluator.py", line 599, in \n [train_losses[i][str(metric)] for i in range(self.num_cv_folds)],\nKeyError: 'average_precision'\n', 'error': "KeyError('average_precision')", 'configuration_origin': 'DUMMY'}.',)

完整代码如下：

import pandas as pd
import category_encoders as ce
from autosklearn.classification import AutoSklearnClassifier
from sklearn.model_selection import train_test_split, StratifiedKFold
from autosklearn.metrics import (accuracy,
                                 f1,
                                 roc_auc,
                                 precision,
                                 average_precision,
                                 recall,
                                 log_loss)
  

df = pd.read_csv('iris.csv')

df['variety'] = df['variety'].astype('category')

y = df.pop('variety')
X = df.copy()

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state=1, stratify=y)

skf = StratifiedKFold(n_splits=5)
  
clf = AutoSklearnClassifier(time_left_for_this_task=600,
                            max_models_on_disc=5,
                            memory_limit = 10240,
                            resampling_strategy=skf,
                            ensemble_size = 3,
                            metric = average_precision,
                            scoring_functions=[roc_auc, average_precision, accuracy, f1, precision, recall, log_loss])    

clf.fit(X = X_train, y = y_train)

曾尝试修改目标变量编码方式：

df['variety'] = df['variety'].apply(pd.Categorical)

原因与解决方案 #

核心原因 #

鸢尾花数据集是多分类任务（3个类别），而你指定的average_precision和roc_auc默认仅支持二分类任务。auto-sklearn的dummy模型在计算多分类任务的这类指标时无法生成对应结果，直接触发KeyError。同时你将average_precision设为主要优化指标，导致dummy模型运行时直接崩溃。

目标变量的编码方式没有问题，两种写法都能正确处理类别标签。

解决方案 #

方案1：更换为多分类兼容的指标

将主要指标和评估函数替换为支持多分类的版本，示例代码如下：

import pandas as pd
import category_encoders as ce
from autosklearn.classification import AutoSklearnClassifier
from sklearn.model_selection import train_test_split, StratifiedKFold
from autosklearn.metrics import (accuracy,
                                 f1_macro,
                                 roc_auc_ovr,
                                 precision_macro,
                                 recall_macro,
                                 log_loss)
  

df = pd.read_csv('iris.csv')

df['variety'] = df['variety'].astype('category')

y = df.pop('variety')
X = df.copy()

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state=1, stratify=y)

skf = StratifiedKFold(n_splits=5)
  
clf = AutoSklearnClassifier(time_left_for_this_task=600,
                            max_models_on_disc=5,
                            memory_limit = 10240,
                            resampling_strategy=skf,
                            ensemble_size = 3,
                            metric = accuracy,  # 替换为多分类兼容的主要指标
                            scoring_functions=[roc_auc_ovr, accuracy, f1_macro, precision_macro, recall_macro, log_loss])    

clf.fit(X = X_train, y = y_train)

• roc_auc_ovr：采用One-vs-Rest模式计算多分类ROC AUC
• f1_macro/precision_macro/recall_macro：多分类场景下的宏平均指标
• accuracy和log_loss本身支持多分类任务

方案2：显式指定多分类模式的平均精度

若需保留average_precision，可使用average_precision_ovr（One-vs-Rest模式），但需注意部分模型可能对该指标的支持有限，优先推荐方案1。

内容的提问来源于stack exchange，提问作者Minura Punchihewa