如何解决布尔型分类特征处理中的Columns must be same length as key错误？

错误原因 #

1. OneHotEncoder输出列数不匹配：OneHotEncoder会将单个分类特征（比如EthnicGroup）根据类别数量扩展为多列，但你直接将这些多列数据赋值回原DataFrame的单列表格，列数不匹配触发报错。
2. 布尔型特征处理同理：Gender、LunchType、TestPrep三个特征经过OneHotEncoder后会生成远多于3列的结果，赋值回原3列的位置必然列数不匹配。
3. 测试集输入错误：布尔型特征处理时，测试集用了x_train作为输入，应该改为x_test。

解决方法 #

方法1：手动处理OneHotEncoder输出并合并到DataFrame #

将OneHotEncoder的输出转为DataFrame，指定列名后，删除原特征列并合并新生成的编码列。

方法2：使用ColumnTransformer统一管理预处理管道（推荐） #

scikit-learn的ColumnTransformer可以针对不同列应用不同的预处理管道，自动处理特征合并，避免手动赋值的错误。

修正后的代码 #

方式1：手动修正版 #

import pandas as pd
from sklearn.impute import SimpleImputer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.pipeline import Pipeline

data = pd.read_csv('Datasets/StudentScore.csv')

target = 'MathScore'
x = data.drop([target, 'Unnamed: 0'], axis=1)
y = data[target]
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=0)

# 数值特征处理
num_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

x_train[['ReadingScore', 'WritingScore']] = num_transformer.fit_transform(x_train[['ReadingScore', 'WritingScore']])
x_test[['ReadingScore', 'WritingScore']] = num_transformer.transform(x_test[['ReadingScore', 'WritingScore']])

# 序数特征处理
education_levels = ["high school", "some high school", "some college", "associate's degree", "bachelor's degree",
                    "master's degree"]

ord_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('encoder', OrdinalEncoder(categories=[education_levels])),
])

x_train[['ParentEduc']] = ord_transformer.fit_transform(x_train[['ParentEduc']])
x_test[['ParentEduc']] = ord_transformer.transform(x_test[['ParentEduc']])

# 名义特征处理
nom_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('encoder', OneHotEncoder(sparse_output=False, drop='first'))  # drop='first'避免多重共线性
])

# 生成编码后的DataFrame并合并
nom_encoded_train = pd.DataFrame(
    nom_transformer.fit_transform(x_train[['EthnicGroup']]),
    columns=nom_transformer.named_steps['encoder'].get_feature_names_out(['EthnicGroup']),
    index=x_train.index
)
x_train = x_train.drop('EthnicGroup', axis=1).join(nom_encoded_train)

nom_encoded_test = pd.DataFrame(
    nom_transformer.transform(x_test[['EthnicGroup']]),
    columns=nom_transformer.named_steps['encoder'].get_feature_names_out(['EthnicGroup']),
    index=x_test.index
)
x_test = x_test.drop('EthnicGroup', axis=1).join(nom_encoded_test)

# 布尔/二元特征处理
bool_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('encoder', OneHotEncoder(sparse_output=False, drop='first'))
])

# 生成编码后的DataFrame并合并
bool_encoded_train = pd.DataFrame(
    bool_transformer.fit_transform(x_train[['Gender', 'LunchType', 'TestPrep']]),
    columns=bool_transformer.named_steps['encoder'].get_feature_names_out(['Gender', 'LunchType', 'TestPrep']),
    index=x_train.index
)
x_train = x_train.drop(['Gender', 'LunchType', 'TestPrep'], axis=1).join(bool_encoded_train)

bool_encoded_test = pd.DataFrame(
    bool_transformer.transform(x_test[['Gender', 'LunchType', 'TestPrep']]),  # 修正为x_test
    columns=bool_transformer.named_steps['encoder'].get_feature_names_out(['Gender', 'LunchType', 'TestPrep']),
    index=x_test.index
)
x_test = x_test.drop(['Gender', 'LunchType', 'TestPrep'], axis=1).join(bool_encoded_test)

方式2：使用ColumnTransformer的规范版（推荐） #

import pandas as pd
from sklearn.impute import SimpleImputer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer

data = pd.read_csv('Datasets/StudentScore.csv')

target = 'MathScore'
x = data.drop([target, 'Unnamed: 0'], axis=1)
y = data[target]
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=0)

# 定义各类型特征列
num_features = ['ReadingScore', 'WritingScore']
ord_features = ['ParentEduc']
nom_features = ['EthnicGroup']
bool_features = ['Gender', 'LunchType', 'TestPrep']

# 数值特征管道
num_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

# 序数特征管道
education_levels = ["high school", "some high school", "some college", "associate's degree", "bachelor's degree",
                    "master's degree"]
ord_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('encoder', OrdinalEncoder(categories=[education_levels]))
])

# 名义特征管道
nom_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('encoder', OneHotEncoder(sparse_output=False, drop='first'))
])

# 布尔特征管道
bool_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('encoder', OneHotEncoder(sparse_output=False, drop='first'))
])

# 统一整合所有预处理管道
preprocessor = ColumnTransformer(
    transformers=[
        ('num', num_transformer, num_features),
        ('ord', ord_transformer, ord_features),
        ('nom', nom_transformer, nom_features),
        ('bool', bool_transformer, bool_features)
    ])

# 直接处理训练集和测试集，输出为numpy数组（如需DataFrame可后续转换）
x_train_processed = preprocessor.fit_transform(x_train)
x_test_processed = preprocessor.transform(x_test)

# 可选：转换为DataFrame查看
feature_names = num_features + ord_features + \
                list(preprocessor.named_transformers_['nom'].named_steps['encoder'].get_feature_names_out(nom_features)) + \
                list(preprocessor.named_transformers_['bool'].named_steps['encoder'].get_feature_names_out(bool_features))
x_train_df = pd.DataFrame(x_train_processed, columns=feature_names, index=x_train.index)
x_test_df = pd.DataFrame(x_test_processed, columns=feature_names, index=x_test.index)

内容的提问来源于stack exchange，提问作者thành nguyễn