处理住房数据集时Pipeline抛出ValueError解包值不足问题求助

问题场景 #

处理住房数据集时，运行代码到pipeline.fit(house_df)（第53行）时抛出以下错误：

ValueError: not enough values to unpack (expected 3, got 2)

原始代码 #

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
from sklearn.pipeline import Pipeline

house_df = pd.read_csv(r"….path….\House_Data.csv")
print(house_df.head(10))
print(house_df.shape)
print(house_df.isnull().sum())
print(house_df.isnull().mean() * 100)
print(house_df['balcony'].mode())
print(house_df['balcony'].median())

imputer1 = house_df['location'].fillna('Whitefield', inplace=True)
imputer2 = house_df['size'].fillna('2 BHK', inplace=True)
imputer3 = house_df['society'].fillna('Others', inplace=True)
imputer4 = house_df['bath'].fillna('2', inplace=True)

for ind, val in enumerate(house_df['total_sqft']):
   if '-' in val:
       temp = val.split('-')
       new_value = (float(temp[0]) + float(temp[-1])) / 2
       house_df['total_sqft'].iloc[ind, :] = new_value

imputer5 = new_value

trf1 = ColumnTransformer(transformers=[
   ('imputer1', imputer1),
   ('imputer2', imputer2),
   ('imputer3', imputer3),
   ('imputer4', imputer4),
   ('imputer5', imputer5)
],
    remainder='passthrough'
)
ohe = OneHotEncoder(handle_unknown='error', drop='first', sparse=False), ["area_type", "location", "size", "society"]

trf2 = ColumnTransformer(transformers=[
     ('ohe', ohe)],
     remainder='passthrough')

pipeline = Pipeline([
     ('trf1', trf1),
     ('trf2', trf2),
  ])
pipeline.fit(house_df) # This is line 53

错误堆栈 #

File "….path…./HousePricePrediction/test3.py", line 53, in
pipeline.fit_transform(house_df)
File "….path….\venv\lib\site-packages\sklearn\pipeline.py", line 426, in fit_transform
Xt = self.fit(X, y, **fit_params_steps) 
File "….path….\HousePricePrediction\venv\lib\site-packages\sklearn\pipeline.py", line 355, in _fit 
*fit_params_steps[name], 
File "….path….\HousePricePrediction\venv\lib\site-packages\joblib\memory.py", line 349, in call 
return self.func(args, **kwargs)
File "….path….\HousePricePrediction\venv\lib\site-packages\sklearn\pipeline.py", line 893, in _fit_transform_one
res = transformer.fit_transform(X, y, *fit_params) 
File "….path….\HousePricePrediction\venv\lib\site-packages\sklearn\compose_column_transformer.py", line 671, in fit_transform 
self._validate_transformers() 
File "….path….\HousePricePrediction\venv\lib\site-packages\sklearn\compose_column_transformer.py", line 324, in _validate_transformers 
names, transformers, = zip(self.transformers)
**ValueError: not enough values to unpack (expected 3, got 2)**

错误原因分析 #

1. ColumnTransformer格式不符合要求：
   每个transformer条目必须是**(名称, 转换器对象, 目标列名)**的三元组，你提供的是二元组（缺少列名），导致Sklearn无法解析。
2. 无效的转换器赋值：
   使用fillna(inplace=True)时，方法返回None，你把这些None赋值给imputer1-4并当作转换器传入，这是完全错误的。
3. OneHotEncoder定义错误：
   错误地用逗号将编码器和列名绑定成元组，导致传入ColumnTransformer的是无效对象。
4. total_sqft处理逻辑错误：
   • iloc[ind, :]适用于DataFrame行切片，但total_sqft是Series，应改为iloc[ind]。
   • 未处理total_sqft中非字符串的情况，可能引发类型错误。
   • imputer5 = new_value无意义，new_value只是循环最后一次计算的值，不是有效转换器。

修正后的代码 #

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.base import BaseEstimator, TransformerMixin

# 自定义转换器处理total_sqft的范围值
class TotalSqftTransformer(BaseEstimator, TransformerMixin):
    def fit(self, X, y=None):
        return self
    
    def transform(self, X):
        def process_sqft(val):
            if isinstance(val, str) and '-' in val:
                temp = val.split('-')
                return (float(temp[0]) + float(temp[-1])) / 2
            try:
                return float(val)
            except:
                return None  # 处理其他非数值格式
        
        X['total_sqft'] = X['total_sqft'].apply(process_sqft)
        return X

house_df = pd.read_csv(r"….path….\House_Data.csv")

# 1. 定义预处理用的缺失值填充器
numeric_imputer = SimpleImputer(strategy='median')
categorical_imputer = SimpleImputer(strategy='constant', fill_value='Others')

# 2. 构建缺失值处理的ColumnTransformer
trf1 = ColumnTransformer(transformers=[
    ('location_impute', categorical_imputer, ['location']),
    ('size_impute', categorical_imputer, ['size']),
    ('society_impute', categorical_imputer, ['society']),
    ('bath_impute', SimpleImputer(strategy='constant', fill_value=2), ['bath']),
    ('balcony_impute', numeric_imputer, ['balcony']),
    ('total_sqft_process', TotalSqftTransformer(), ['total_sqft'])
], remainder='passthrough')

# 3. 定义编码步骤的ColumnTransformer
ohe = OneHotEncoder(handle_unknown='ignore', drop='first', sparse_output=False)
trf2 = ColumnTransformer(transformers=[
    ('ohe_encode', ohe, ['area_type', 'location', 'size', 'society'])
], remainder='passthrough')

# 4. 构建Pipeline并训练
pipeline = Pipeline([
    ('trf1', trf1),
    ('trf2', trf2),
])

pipeline.fit(house_df)

关键修正点说明 #

• 用SimpleImputer替代手动fillna：确保转换器是Sklearn兼容对象，能融入Pipeline流程。
• 自定义TotalSqftTransformer：专门处理total_sqft的范围值，同时兼容非数值格式。
• 严格遵循ColumnTransformer格式：每个条目都包含名称、转换器、列名三元组。
• 修正OneHotEncoder定义：单独定义编码器，再在ColumnTransformer中指定目标列。

内容的提问来源于stack exchange，提问作者Sam2021