sklearn中VotingClassifier结合RandomizedSearchCV遇pickle序列化错误求助

问题 #

在sklearn 1.1.3版本中，尝试将随机超参数搜索（RandomizedSearchCV）与VotingClassifier结合使用时，运行代码出现cannot pickle 'dict_items' object错误，使用GridSearchCV也会触发相同问题。

简化代码如下：

import numpy as np
from sklearn import __version__ as skv
from sklearn.ensemble import RandomForestClassifier as RFClassi
from sklearn.ensemble import HistGradientBoostingClassifier as HGBClassi
from sklearn.tree import DecisionTreeClassifier as DTClassi
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import VotingClassifier
from sklearn.datasets import load_iris

print(f"sklearn version: {skv}")

df_X, target = load_iris(return_X_y=True, as_frame=True)
ensemble = ['rf','dtree','hgb']
hy_pa_grid = {
    'hgb': dict(learning_rate = list(np.linspace(0.01,0.5,10).round(3))),
    'rf':dict(criterion = ['gini', 'entropy']),
    'dtree':dict(criterion = ['gini', 'entropy']),
}
clfs = {'hgb' : HGBClassi(), 'rf': RFClassi(), 'dtree' : DTClassi()}
vc = VotingClassifier(estimators = clfs.items(), voting = 'soft')
params = {
    f"{c}__{p}" : hy_pa_grid[c][p]
    for c in ensemble
    for p in hy_pa_grid[c].keys()
}
print("\n".join(map(str,params.items())))
clf = RandomizedSearchCV(estimator = vc, param_distributions = params)
clf.fit(df_X,target)

运行输出：

sklearn version: 1.1.3
{'rf__criterion': ['gini', 'entropy'], 'dtree__criterion': ['gini', 'entropy'], 'hgb__learning_rate': [0.01, 0.064, 0.119, 0.173, 0.228, 0.282, 0.337, 0.391, 0.446, 0.5]}
Traceback (most recent call last):
  File "vc.py", line 34, in <module>
    clf.fit(df_X,target)                
  File "/home/USER/.local/lib/python3.8/site-packages/sklearn/model_selection/_search.py", line 789, in fit
    base_estimator = clone(self.estimator)
  File "/home/USER/.local/lib/python3.8/site-packages/sklearn/base.py", line 87, in clone
    new_object_params[name] = clone(param, safe=False)
  File "/home/USER/.local/lib/python3.8/site-packages/sklearn/base.py", line 68, in clone
    return copy.deepcopy(estimator)
  File "/usr/lib/python3.8/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle 'dict_items' object

解决办法 #

错误根源 #

clfs.items()返回的是dict_items对象，这是一种迭代器类型，无法被Python的pickle模块序列化。而sklearn的交叉验证类（如RandomizedSearchCV）在执行fit时，需要克隆传入的estimator，这个过程依赖深拷贝操作，因此触发序列化错误。

修复步骤 #

将clfs.items()转换为列表——列表是可序列化的容器类型，能够被深拷贝正常处理。

修改这一行代码：

vc = VotingClassifier(estimators = clfs.items(), voting = 'soft')

改为：

vc = VotingClassifier(estimators = list(clfs.items()), voting = 'soft')

修改后的完整代码 #

import numpy as np
from sklearn import __version__ as skv
from sklearn.ensemble import RandomForestClassifier as RFClassi
from sklearn.ensemble import HistGradientBoostingClassifier as HGBClassi
from sklearn.tree import DecisionTreeClassifier as DTClassi
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import VotingClassifier
from sklearn.datasets import load_iris

print(f"sklearn version: {skv}")

df_X, target = load_iris(return_X_y=True, as_frame=True)
ensemble = ['rf','dtree','hgb']
hy_pa_grid = {
    'hgb': dict(learning_rate = list(np.linspace(0.01,0.5,10).round(3))),
    'rf':dict(criterion = ['gini', 'entropy']),
    'dtree':dict(criterion = ['gini', 'entropy']),
}
clfs = {'hgb' : HGBClassi(), 'rf': RFClassi(), 'dtree' : DTClassi()}
# 将dict_items转换为列表
vc = VotingClassifier(estimators = list(clfs.items()), voting = 'soft')
params = {
    f"{c}__{p}" : hy_pa_grid[c][p]
    for c in ensemble
    for p in hy_pa_grid[c].keys()
}
print("\n".join(map(str,params.items())))
clf = RandomizedSearchCV(estimator = vc, param_distributions = params)
clf.fit(df_X,target)

运行修改后的代码，即可正常执行随机超参数搜索，不会再触发pickle错误。

内容的提问来源于stack exchange，提问作者Mr Felix U