XGBRegressor Pipeline中early_stopping_rounds参数报错解决咨询

问题场景 #

预处理CSV数据后，尝试通过early_stopping_rounds让XGBoost自动选择最优n_estimators，但运行时抛出类型错误；但不启用早停机制时，模型能正常训练并输出结果。

报错代码 #

# Preprocessing for numerical data
numerical_transformer = SimpleImputer(strategy='constant')

# Preprocessing for categorical data
categorical_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('onehot', OneHotEncoder(handle_unknown='ignore', dtype=int))
])

from sklearn.compose import ColumnTransformer

numerical_cols = home_data.select_dtypes(include='number').columns.tolist()
categorical_cols = home_data.select_dtypes(include=['object', 'category']).columns.tolist()

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numerical_transformer, numerical_cols),
        ('cat', categorical_transformer, categorical_cols)
    ])

from xgboost import XGBRegressor

model = XGBRegressor(n_estimators=1000, learning_rate=0.05, n_jobs=4, enable_categorical=True)

my_pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('model', model)
])

# Fit model and evaluate
my_pipeline.fit(train_X, train_y, model__eval_set=[(val_X, val_y)], model__early_stopping_rounds=5, model__verbose=False)

报错信息 #

ValueError: DataFrame.dtypes for data must be int, float, bool or category. When categorical type is supplied, The experimental DMatrix parameter`enable_categorical` must be set to `True`.  Invalid columns:MSZoning: object, Street: object, Alley: object, LotShape: object, LandContour: object, Utilities: object, LotConfig: object, LandSlope: object, Neighborhood: object, Condition1: object, Condition2: object, BldgType: object, HouseStyle: object, RoofStyle: object, RoofMatl: object, Exterior1st: object, Exterior2nd: object, MasVnrType: object, ExterQual: object, ExterCond: object, Foundation: object, BsmtQual: object, BsmtCond: object, BsmtExposure: object, BsmtFinType1: object, BsmtFinType2: object, Heating: object, HeatingQC: object, CentralAir: object, Electrical: object, KitchenQual: object, Functional: object, FireplaceQu: object, GarageType: object, GarageFinish: object, GarageQual: object, GarageCond: object, PavedDrive: object, PoolQC: object, Fence: object, MiscFeature: object, SaleType: object, SaleCondition: object

正常运行的代码（无早停） #

from xgboost import XGBRegressor

model = XGBRegressor(n_estimators=220)

#model = XGBRegressor(n_estimators=1000, learning_rate=0.05, n_jobs=4, enable_categorical=True)


my_pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('model', model)
])

my_pipeline.fit(train_X, train_y)

#my_pipeline.fit(train_X, train_y, model__eval_set=[(val_X, val_y)], model__early_stopping_rounds=5, model__verbose=False)

preds = my_pipeline.predict(val_X)
score = mean_absolute_error(val_y, preds)
print('MAE:', score)

正常运行结果 #

MAE: 1739.843953339041

报错原因 #

核心问题是传给eval_set的验证数据val_X没有经过预处理：

• Pipeline的fit方法会自动用preprocessor处理train_X，但eval_set里的val_X是直接传入XGBoost的，仍然保留原始的object类型列；
• 同时你开启了enable_categorical=True，XGBoost会严格检查输入数据类型，导致格式不匹配的报错；
• 另外，enable_categorical=True是多余的——你已经对分类特征做了OneHot编码，不需要XGBoost的原生分类特征支持，反而会引发类型检查冲突。

解决方案 #

修改后的完整代码 #

# Preprocessing for numerical data
numerical_transformer = SimpleImputer(strategy='constant')

# Preprocessing for categorical data
categorical_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('onehot', OneHotEncoder(handle_unknown='ignore', dtype=int))
])

from sklearn.compose import ColumnTransformer
from sklearn.metrics import mean_absolute_error

numerical_cols = home_data.select_dtypes(include='number').columns.tolist()
categorical_cols = home_data.select_dtypes(include=['object', 'category']).columns.tolist()

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numerical_transformer, numerical_cols),
        ('cat', categorical_transformer, categorical_cols)
    ])

from xgboost import XGBRegressor

# 移除enable_categorical=True，因为已做OneHot编码
model = XGBRegressor(n_estimators=1000, learning_rate=0.05, n_jobs=4)

my_pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('model', model)
])

# 先预处理验证集，确保和训练数据格式一致
val_X_processed = preprocessor.transform(val_X)

# 启用早停训练模型
my_pipeline.fit(train_X, train_y, 
                model__eval_set=[(val_X_processed, val_y)], 
                model__early_stopping_rounds=5, 
                model__verbose=False)

# 预测时直接用Pipeline处理验证集
preds = my_pipeline.predict(val_X)
score = mean_absolute_error(val_y, preds)
print('MAE:', score)

关键修改点 #

• 预处理验证集：用preprocessor.transform(val_X)将验证数据转换成和训练数据一致的数值化格式；
• 移除多余参数：删除enable_categorical=True，避免XGBoost对分类特征的类型检查冲突；
• 保持Pipeline完整性：预测时仍用my_pipeline.predict(val_X)，让Pipeline自动处理验证数据。

内容的提问来源于stack exchange，提问作者Kawith