OneHotEncoder致GridSearch特征数不匹配问题排查与修复

问题描述 #

构建了如下机器学习预处理管道：

#from sklearn import set_config
#set_config(transform_output='pandas')

class Preprocessor(TransformerMixin):
    def __init__(self):
        self._cluster_simil = ClusterSimilarity(n_clusters=10, gamma=1., random_state=42)
        #pass

    def fit(self, X, y=None):
        return self

    def transform(self, X):
        #self._cluster_simil = ClusterSimilarity(n_clusters=10, gamma=1., random_state=42)
        preprocessing = self._preprocessing()
        return preprocessing.fit_transform(X)

    def _column_ratio(self, X):
        return X[:, [0]] / X[:, [1]]

    def _ratio_name(self, function_transformer, feature_names_in):
        return ["ratio"]

    def _ratio_pipeline(self):
        return make_pipeline(
            SimpleImputer(strategy="median"),
            FunctionTransformer(self._column_ratio, feature_names_out=self._ratio_name),
            StandardScaler()
        )

    def _log_pipeline(self):
        return make_pipeline(
            SimpleImputer(strategy="median"),
            FunctionTransformer(np.log, feature_names_out="one-to-one"),
            StandardScaler()
        )

    def _cat_pipeline(self):
        return make_pipeline(
            SimpleImputer(strategy="most_frequent"),
            OneHotEncoder(handle_unknown="ignore")
        )

    def _default_num_pipeline(self):
        return make_pipeline(SimpleImputer(strategy="median"),
                             StandardScaler()
        )

    def _preprocessing(self):
        return ColumnTransformer([
            ("bedrooms", self._ratio_pipeline(), ["total_bedrooms", "total_rooms"]),
            ("rooms_per_house", self._ratio_pipeline(), ["total_rooms", "households"]),
            ("people_per_house", self._ratio_pipeline(), ["population", "households"]),
            ("log", self._log_pipeline(), ["total_bedrooms", "total_rooms", "population",
                                "households", "median_income"]),
            ("geo", self._cluster_simil, ["latitude", "longitude"]),
            ("cat", self._cat_pipeline(), make_column_selector(dtype_include=object)),
    ],
    remainder=self._default_num_pipeline())  # one column remaining: housing_median_age

class ClusterSimilarity(BaseEstimator, TransformerMixin):
    def __init__(self, n_clusters=10, gamma=1.0, random_state=None):
        self.n_clusters = n_clusters
        self.gamma = gamma
        self.random_state = random_state

    def fit(self, X, y=None, sample_weight=None):
        self.kmeans_ = KMeans(self.n_clusters, n_init=10,
                              random_state=self.random_state)
        self.kmeans_.fit(X, sample_weight=sample_weight)
        return self  # always return self!

    def transform(self, X):
        return rbf_kernel(X, self.kmeans_.cluster_centers_, gamma=self.gamma)

    def get_feature_names_out(self, names=None):
        return [f"Cluster {i} similarity" for i in range(self.n_clusters)]

单独测试预处理管道时，X_train.info()显示特征数为24（符合预期），但执行GridSearch代码时：

svr_pipeline = Pipeline([("preprocessing", preprocessor), ("svr", SVR())])
grid_search = GridSearchCV(svr_pipeline, param_grid, cv=3,
                           scoring='neg_root_mean_squared_error')
                       
grid_search.fit(housing.iloc[:5000], housing_labels.iloc[:5000])

出现报错：

ValueError: The feature names should match those that were passed during fit.
Feature names seen at fit time, yet now missing:
- cat__ocean_proximity_ISLAND

UserWarning: One or more of the test scores are non-finite:
[nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan
 nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan
 nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan
 nan nan nan nan nan nan nan nan]

核心问题为：ValueError: X has 23 features, but SVR is expecting 24 features as input，缺失的特征cat__ocean_proximity_ISLAND来自OneHotEncoder处理的类别特征，该类别样本量极少。仅当数据集包含该类别样本时才会触发报错，小数据集测试无异常。

问题分析 #

• 问题根源并非OneHotEncoder本身，而是Preprocessor类的实现错误：
  当前Preprocessor的fit方法仅返回self，未完成任何预处理逻辑的拟合；而transform方法每次调用都会重新创建ColumnTransformer并执行fit_transform(X)，这意味着每次转换都会重新拟合所有预处理组件（包括OneHotEncoder、KMeans等），完全违背了sklearn的fit-then-transform范式。

• 仅在GridSearch时报错的原因：
  GridSearch采用交叉验证（CV）会将数据集拆分为多个训练/测试折。当某个稀有类别（如ISLAND）仅出现在部分折的训练集中时，该折的OneHotEncoder会生成对应特征；但其他折的训练集没有该类别，OneHotEncoder就不会生成这个特征。或者当某折的训练集包含该类别，但测试集没有时，测试集预处理后的特征数会少于训练集，导致SVR因输入特征数量不匹配报错。而单独测试时，是对整个数据集执行fit_transform，所有类别都被捕获，特征数一致，因此无异常。

修复与规避方案 #

1. 修正Preprocessor类的实现（核心修复） #

遵循sklearn Estimator的标准范式，在fit方法中完成内部预处理管道的拟合，transform方法仅执行转换操作：

class Preprocessor(BaseEstimator, TransformerMixin):  # 需继承BaseEstimator
    def __init__(self):
        self._cluster_simil = ClusterSimilarity(n_clusters=10, gamma=1., random_state=42)
        self._preprocessor = None  # 存储拟合后的预处理管道

    def fit(self, X, y=None):
        # 在fit方法中创建并拟合预处理管道
        self._preprocessor = self._preprocessing()
        self._preprocessor.fit(X, y)
        return self

    def transform(self, X):
        # 仅使用已拟合的管道执行转换
        return self._preprocessor.transform(X)

    # 其余方法（_column_ratio、_ratio_name等）保持不变

注意：Preprocessor需要继承BaseEstimator，才能被sklearn的Pipeline正确识别。

2. 固定OneHotEncoder的类别（可选增强） #

为了彻底避免因训练集类别不全导致的特征数变化，可以提前获取所有可能的类别，手动指定给OneHotEncoder：

def _cat_pipeline(self):
    # 假设提前获取了ocean_proximity的所有类别：['NEAR BAY', 'INLAND', 'NEAR OCEAN', 'ISLAND', '<1H OCEAN']
    return make_pipeline(
        SimpleImputer(strategy="most_frequent"),
        OneHotEncoder(handle_unknown="ignore", categories=[['NEAR BAY', 'INLAND', 'NEAR OCEAN', 'ISLAND', '<1H OCEAN']])
    )

这样即使训练集缺失某个类别，OneHotEncoder也会保留对应特征的位置（值为0），确保特征数始终一致。

3. 其他注意事项 #

• 确保所有自定义Transformer都继承BaseEstimator和TransformerMixin，符合sklearn的接口规范。
• 交叉验证时，预处理逻辑必须在每个折的训练集上拟合，然后应用到对应测试集，修正后的Preprocessor会自动遵循这个逻辑，避免特征空间不一致。

内容的提问来源于stack exchange，提问作者Silvio sjsj