如何在Sklearn中直接使用字符串数据进行预测？

问题根源 #

训练时用LabelEncoder将字符串特征转为数字，但测试输入的原始字符串未经过和训练时完全一致的编码转换，模型只能处理数值型特征，因此抛出类型错误。此外，你的代码中复用了同一个LabelEncoder实例，会导致前一个特征的编码规则被覆盖，这是潜在bug。

方案1：为每个字符串特征单独保存编码器，测试时手动转换 #

修改训练代码，为product和Brand分别创建独立的编码器并保存，测试时用对应的编码器转换字符串特征：

训练代码修改：

import sqlalchemy
import pandas as pd
read_engine=sqlalchemy.create_engine('mysql+mysqlconnector://root:@localhost/six')
conn = read_engine.connect()
df_new=pd.read_sql_table('mobile1' ,con= conn )
# 数据预处理部分不变
df_new['price']=df_new['price'].astype(int)
df_new['ram']=df_new['ram'].astype(int)
df_new['battery']=df_new['battery'].astype(int)
df_new['size']=df_new['size'].astype(float)
df_new['camera']=df_new['camera'].mask(df_new['camera'] == '')
df_new['camera']=df_new['camera'].mask(df_new['camera'] == ' ')
df_new['camera']=df_new['camera'].mask(df_new['camera'] == '  ')
df_new['camera']=df_new['camera'].fillna(0)
df_new['camera']=df_new['camera'].astype(float)

X=df_new[['ram','size','camera','product','Brand','battery']]
Y=df_new['price'].values

from sklearn import preprocessing
# 为每个字符串特征创建独立编码器
product_enc = preprocessing.LabelEncoder()
X['product'] = product_enc.fit_transform(X['product'])

brand_enc = preprocessing.LabelEncoder()
X['Brand'] = brand_enc.fit_transform(X['Brand'])

from sklearn import tree
dtc=tree.DecisionTreeClassifier()
learn_to_machine=dtc.fit(X.values,Y)

测试代码：

# 原始字符串测试数据
test=[
    [128, 6 ,50, 'mobile_phone', 'Samsung', 6000],
    [512, 8, 65, 'mobile_phone', 'Huawei',5000]
]
# 将测试数据转为DataFrame，方便按特征处理
test_df = pd.DataFrame(test, columns=['ram','size','camera','product','Brand','battery'])
# 用保存的编码器转换字符串特征
test_df['product'] = product_enc.transform(test_df['product'])
test_df['Brand'] = brand_enc.transform(test_df['Brand'])

# 执行预测
answer=learn_to_machine.predict(test_df.values)
print(answer)

方案2：使用Pipeline+ColumnTransformer实现自动化预处理（推荐） #

用Sklearn的ColumnTransformer指定不同特征的预处理规则，再结合Pipeline将预处理和模型串联，训练和测试全程无需手动编码：

完整代码：

import sqlalchemy
import pandas as pd
from sklearn import preprocessing
from sklearn import tree
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline

read_engine=sqlalchemy.create_engine('mysql+mysqlconnector://root:@localhost/six')
conn = read_engine.connect()
df_new=pd.read_sql_table('mobile1' ,con= conn )
# 数据预处理部分不变
df_new['price']=df_new['price'].astype(int)
df_new['ram']=df_new['ram'].astype(int)
df_new['battery']=df_new['battery'].astype(int)
df_new['size']=df_new['size'].astype(float)
df_new['camera']=df_new['camera'].mask(df_new['camera'] == '')
df_new['camera']=df_new['camera'].mask(df_new['camera'] == ' ')
df_new['camera']=df_new['camera'].mask(df_new['camera'] == '  ')
df_new['camera']=df_new['camera'].fillna(0)
df_new['camera']=df_new['camera'].astype(float)

X=df_new[['ram','size','camera','product','Brand','battery']]
Y=df_new['price'].values

# 定义预处理规则：字符串特征用LabelEncoder，数值特征保持原样
numeric_features = ['ram','size','camera','battery']
categorical_features = ['product','Brand']

preprocessor = ColumnTransformer(
    transformers=[
        ('cat', preprocessing.LabelEncoder(), categorical_features)
    ],
    remainder='passthrough'  # 数值特征直接保留
)

# 构建Pipeline：预处理 + 模型
pipe = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('classifier', tree.DecisionTreeClassifier())
])

# 训练模型
pipe.fit(X, Y)

# 直接用原始字符串测试数据预测
test=[
    [128, 6 ,50, 'mobile_phone', 'Samsung', 6000],
    [512, 8, 65, 'mobile_phone', 'Huawei',5000]
]
# 转为DataFrame保证列名和训练时一致
test_df = pd.DataFrame(test, columns=['ram','size','camera','product','Brand','battery'])
answer = pipe.predict(test_df)
print(answer)

内容的提问来源于stack exchange，提问作者M.Namjoo