使用sklearn.SimpleImputer填充字符串列缺失值时遇ValueError问题
问题描述
我有一个包含缺失值(NA)的DataFrame列列表,所有列的dtype均为str:
X_train_objects = ['HomePlanet', 'Destination', 'Name', 'Cabin_letter', 'Cabin_number', 'Cabin_letter_2']
我想用SimpleImputer以众数(most frequent)填充这些缺失值,但运行代码时收到报错:ValueError: Columns must be same length as key。
DataFrame(X_train)样例(其中Destination列全为np.NA):
{'PassengerId': {47: '0045_02', 128: '0138_02', 139: '0152_01', 347: '0382_01', 430: '0462_01'}, 'HomePlanet': {47: 'Mars', 128: 'Earth', 139: 'Earth', 347: nan, 430: 'Earth'}, 'CryoSleep': {47: 1, 128: 0, 139: 0, 347: 0, 430: 1}, 'Destination': {47: nan, 128: nan, 139: nan, 347: nan, 430: nan}, 'Age': {47: 19.0, 128: 34.0, 139: 41.0, 347: 23.0, 430: 50.0}, 'VIP': {47: 0, 128: 0, 139: 0, 347: 0, 430: 0}, 'RoomService': {47: 0.0, 128: 0.0, 139: 0.0, 347: 348.0, 430: 0.0}, 'FoodCourt': {47: 0.0, 128: 22.0, 139: 0.0, 347: 0.0, 430: 0.0}, 'ShoppingMall': {47: 0.0, 128: 0.0, 139: 0.0, 347: 0.0, 430: 0.0}, 'Spa': {47: 0.0, 128: 564.0, 139: 0.0, 347: 4.0, 430: 0.0}, 'VRDeck': {47: 0.0, 128: 207.0, 139: 607.0, 347: 368.0, 430: 0.0}, 'Name': {47: 'Mass Chmad', 128: 'Monah Gambs', 139: 'Andan Estron', 347: 'Blanie Floydendley', 430: 'Ronia Sosanturney'}, 'Transported': {47: 1, 128: 0, 139: 0, 347: 0, 430: 0}, 'Cabin_letter': {47: 'F', 128: 'E', 139: 'F', 347: 'G', 430: 'G'}, 'Cabin_number': {47: '10', 128: '5', 139: '32', 347: '64', 430: '67'}, 'Cabin_letter_2': {47: 'P', 128: 'P', 139: 'P', 347: 'P', 430: 'S'}}
我的代码:
imputer = SimpleImputer(missing_values=np.NaN, strategy='most_frequent') X_train[X_train_objects] = imputer.fit_transform(X_train[X_train_objects].values.reshape(-1,1))[:,0]
报错原因
- 维度不匹配:你将
X_train[X_train_objects]的数组用reshape(-1,1)转换成了单列(行数为样本数×列数),fit_transform后得到的仍是单列数组,再通过[:,0]取出后变成一维数组,长度为样本数×6。但X_train[X_train_objects]需要的是样本数×6列的二维数据,一维数组无法匹配6列的结构,因此抛出“列长度必须与键一致”的错误。 - 全缺失列的潜在问题:
Destination列全为缺失值,most_frequent策略在这种场景下会因没有可提取的众数而报错,这也是后续需要处理的隐藏问题。
修正方案
方案1:直接对多列进行Imputer处理(无需手动reshape)
SimpleImputer原生支持多列数据处理,不需要修改维度:
from sklearn.impute import SimpleImputer import numpy as np imputer = SimpleImputer(missing_values=np.NaN, strategy='most_frequent') # 直接对选中列拟合转换,结果为二维数组,可直接赋值回原DataFrame X_train[X_train_objects] = imputer.fit_transform(X_train[X_train_objects])
方案2:处理全缺失列的特殊情况
针对Destination这类全缺失的列,需要额外指定默认填充值:
# 方式1:逐个列处理 for col in X_train_objects: if X_train[col].isna().all(): X_train[col] = 'Unknown' # 自定义填充值 else: imputer = SimpleImputer(missing_values=np.NaN, strategy='most_frequent') X_train[col] = imputer.fit_transform(X_train[[col]]) # 方式2:利用SimpleImputer的fill_value参数(仅当列全缺失时生效) imputer = SimpleImputer(missing_values=np.NaN, strategy='most_frequent', fill_value='Unknown') X_train[X_train_objects] = imputer.fit_transform(X_train[X_train_objects])
内容的提问来源于stack exchange,提问作者Katsu
相关产品推荐
相关产品推荐

