Pipeline分类数据编码报错：无法将'Average'转为浮点数的排查与解决

问题描述 #

尝试对分类列Sex使用OneHotEncoder编码，Blood Pressure和Diet列使用OrdinalEncoder编码，之后对数据进行缩放，整个流程通过Pipeline实现，但训练时出现以下错误：

ValueError: could not convert string to float: 'Average'

用户的原始代码如下：

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import (
    OrdinalEncoder,
    OneHotEncoder,
    MinMaxScaler,
    StandardScaler,
)
from sklearn.ensemble import RandomForestClassifier
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
import numpy as np
from sklearn import set_config

set_config(display="diagram")
data = pd.read_csv("/content/heart_attack_prediction_dataset.csv")
data.shape
data = data.drop(["Patient ID", "Country", "Continent", "Hemisphere", "Income"], axis=1)


data["Blood Pressure"][0]

bp_list = []
for each in data["Blood Pressure"]:
    bp = each.split("/")
    systolic = int(bp[0])

    diastolic = int(bp[1])

    if systolic < 120 and diastolic < 80:
        bp_list.append("Normal")
    elif systolic > 120 and systolic < 139 and diastolic > 80 and diastolic < 89:
        bp_list.append("Pre Hypertension")
    elif systolic > 140 and systolic < 159 and diastolic > 90 and diastolic < 99:
        bp_list.append("Stage 1 Hypertension")
    else:
        bp_list.append("Stage 2 Hypertension")


data["Blood Pressure"] = bp_list


# Encoding Sex

trf1_ohe = ColumnTransformer(
    [("ohe", OneHotEncoder(sparse_output=False, handle_unknown="ignore"), [1])],
    remainder="passthrough",
)
# Encoding Categorical Data column
trf2_oe = ColumnTransformer(
    [
        (
            "oe",
            OrdinalEncoder(
                categories=[
                    [
                        "Normal",
                        "Pre Hypertension",
                        "Stage 1 Hypertension",
                        "Stage 2 Hypertension",
                    ],
                    ["Good", "Average", "Unhealthy"],
                ],
                handle_unknown="use_encoded_value",
                unknown_value=np.nan,
            ),
            [4, 11],
        )
    ],
    remainder="passthrough",
)

# Scaling

trf3_scal = ColumnTransformer(
    [("scaling", StandardScaler(), [0, 3, 8, 20])], remainder="passthrough"
)

trf4_clf = RandomForestClassifier(n_estimators=100)

pipe = Pipeline(
    [("trf1", trf1_ohe), ("trf2", trf2_oe), ("trf3", trf3_scal), ("trf4", trf4_clf)]
)

pipe

X_train, X_test, y_train, y_test = train_test_split(
    data.drop(columns=["Heart Attack Risk"]),
    data["Heart Attack Risk"],
    test_size=0.3,
    random_state=11,
)

pipe.fit(X_train, y_train)

问题原因 #

1. 列索引混乱：trf2_oe中使用的列索引[4,11]是原始数据的列位置，但经过trf1_ohe处理后，Sex列被OneHotEncoder拆分为多列，后续所有列的位置都会后移，导致[4,11]不再指向Blood Pressure和Diet列，未被正确编码的字符串列进入缩放步骤后，StandardScaler无法处理字符串，从而报错。
2. 串行ColumnTransformer设计不合理：多次串行使用ColumnTransformer会不断改变数据的列结构，很容易出现列映射错误，且维护成本高。

解决方法 #

改用统一的预处理ColumnTransformer，直接用列名指定需要处理的特征，避免索引混乱，同时将所有预处理逻辑整合到一个步骤中：

修改后的代码 #

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import (
    OrdinalEncoder,
    OneHotEncoder,
    StandardScaler,
)
from sklearn.ensemble import RandomForestClassifier
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
import numpy as np
from sklearn import set_config

set_config(display="diagram")
data = pd.read_csv("/content/heart_attack_prediction_dataset.csv")
data = data.drop(["Patient ID", "Country", "Continent", "Hemisphere", "Income"], axis=1)

# 处理Blood Pressure列，修正边界条件
bp_list = []
for each in data["Blood Pressure"]:
    bp = each.split("/")
    systolic = int(bp[0])
    diastolic = int(bp[1])
    if systolic < 120 and diastolic < 80:
        bp_list.append("Normal")
    elif 120 <= systolic <= 139 and 80 <= diastolic <= 89:
        bp_list.append("Pre Hypertension")
    elif 140 <= systolic <= 159 and 90 <= diastolic <= 99:
        bp_list.append("Stage 1 Hypertension")
    else:
        bp_list.append("Stage 2 Hypertension")
data["Blood Pressure"] = bp_list

# 明确各类特征列
ohe_cols = ["Sex"]  # 需要OneHot编码的列
oe_cols = ["Blood Pressure", "Diet"]  # 需要序数编码的列
# 筛选数值列：排除分类列和目标列
num_cols = [col for col in data.columns if col not in ohe_cols + oe_cols + ["Heart Attack Risk"]]

# 构建统一的预处理模块
preprocessor = ColumnTransformer(
    transformers=[
        # OneHot编码处理Sex列
        ("ohe", OneHotEncoder(sparse_output=False, handle_unknown="ignore"), ohe_cols),
        # 序数编码处理血压和饮食列
        ("oe", OrdinalEncoder(
            categories=[
                ["Normal", "Pre Hypertension", "Stage 1 Hypertension", "Stage 2 Hypertension"],
                ["Good", "Average", "Unhealthy"]
            ],
            handle_unknown="use_encoded_value",
            unknown_value=np.nan
        ), oe_cols),
        # 标准化处理数值列
        ("scaler", StandardScaler(), num_cols)
    ])

# 完整流水线：预处理 + 分类器
pipe = Pipeline([
    ("preprocessor", preprocessor),
    ("classifier", RandomForestClassifier(n_estimators=100))
])

# 划分数据集
X_train, X_test, y_train, y_test = train_test_split(
    data.drop(columns=["Heart Attack Risk"]),
    data["Heart Attack Risk"],
    test_size=0.3,
    random_state=11,
)

# 训练模型
pipe.fit(X_train, y_train)

关键修改点 #

1. 用列名替代索引：直接指定列名，彻底避免Pipeline步骤中列位置变化导致的映射错误。
2. 整合预处理逻辑：将OneHot编码、序数编码、数值缩放统一到一个ColumnTransformer中，结构更清晰，也不会出现列混乱。
3. 修正血压判断边界：原来的条件漏掉了等于临界值的情况，现在调整为符合医学标准的区间判断，保证数据处理的准确性。

内容的提问来源于stack exchange，提问作者111