RandomForestClassifier预测结果全为0问题排查求助

问题描述 #

训练RandomForestClassifier模型预测肥料类型，模型输出的y_pred全部为0，数据集约75万行，代码及数据样本如下：

实现代码 #

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

df = pd.read_csv("train.csv")
X_train = df.iloc[:, 1:-1].values
y_train = df.iloc[:, [-1]].values

df = pd.read_csv("test.csv")
X_test = df.iloc[:, 1:].values

from sklearn.impute import SimpleImputer
imputer = SimpleImputer(missing_values=np.nan, strategy = "most_frequent")
imputer.fit(X_train[:, :])
X_train[:, :] = imputer.transform(X_train[:, :])
X_test[:, :] = imputer.transform(X_test[:, :])

int_features = []
categorical_features = []
for i in range(len(X_train[0])) : 
    if type(X_train[0][i]) == int or type(X_train[0][i]) == float : 
        int_features.append(i)
    elif type(X_train[0][i]) == str : 
        categorical_features.append(i)

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
ct_x = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), categorical_features)], remainder='passthrough')
X_train = np.array(ct_x.fit_transform(X_train))
X_test = np.array(ct_x.transform(X_test))

ct_y = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [0])], remainder='passthrough', sparse_threshold=0)
y_train = np.array(ct_y.fit_transform(y_train))

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train[:, int_features] = sc.fit_transform(X_train[:, int_features])
X_test[:, int_features] = sc.transform(X_test[:, int_features])

from sklearn.ensemble import RandomForestClassifier
classifier = RandomForestClassifier(n_estimators=500, max_depth=25, random_state=42)
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)

#print(y_pred)
# Access the OneHotEncoder
ohe = ct_y.named_transformers_['encoder']

# Apply inverse_transform
inverse_transformed_data = ohe.inverse_transform(y_pred)

#print(inverse_transformed_data)
#print(inverse_transformed_data)

数据样本 #

train.csv：

id,Temparature,Humidity,Moisture,Soil Type,Crop Type,Nitrogen,Potassium,Phosphorous,Fertilizer Name
0,37,70,36,Clayey,Sugarcane,36,4,5,28-28
1,27,69,65,Sandy,Millets,30,6,18,28-28
2,29,63,32,Sandy,Millets,24,12,16,17-17-17
3,35,62,54,Sandy,Barley,39,12,4,10-26-26
4,35,58,43,Red,Paddy,37,2,16,DAP
5,30,59,29,Red,Pulses,10,0,9,20-20
6,27,62,53,Sandy,Paddy,26,15,22,28-28
7,36,62,44,Red,Pulses,30,12,35,14-35-14
8,36,51,32,Loamy,Tobacco,19,17,29,17-17-17
9,28,50,35,Red,Tobacco,25,12,16,20-20

test.csv：

id,Temparature,Humidity,Moisture,Soil Type,Crop Type,Nitrogen,Potassium,Phosphorous
750000,31,70,52,Sandy,Wheat,34,11,24
750001,27,62,45,Red,Sugarcane,30,14,15
750002,28,72,28,Clayey,Ground Nuts,14,15,4
750003,37,53,57,Black,Ground Nuts,18,17,36
750004,31,55,32,Red,Pulses,13,19,14
750005,35,63,34,Black,Millets,36,3,2
750006,38,50,56,Clayey,Sugarcane,32,6,31
750007,25,55,44,Black,Barley,32,9,32
750008,29,56,60,Red,Pulses,26,5,13
750009,25,63,40,Loamy,Sugarcane,9,5,41

核心问题分析 #

1. 目标变量编码错误：用OneHotEncoder处理单分类任务 #

对y_train做OneHotEncoder得到二维二进制矩阵，但RandomForestClassifier会将其视为多标签分类任务（每个列独立预测），而非单分类任务。这会导致模型损失计算逻辑错误，学习方向偏离，最终输出异常。单分类任务的目标变量应该用LabelEncoder转成整数索引，而非OneHot编码。

2. 特征预处理索引失效 #

OneHot编码后，X_train的列结构已完全改变：原始分类特征被展开为多列，而仍用原始特征的索引去选取数值特征做标准化，会错误地将OneHot编码后的分类列当成数值特征处理，真正的数值特征却未被标准化，导致特征混乱，模型无法有效学习。

3. 潜在的类别不平衡问题 #

从样本数据看，28-28肥料出现次数最多。若整个75万行数据中某类占比极高（如90%以上），模型会倾向于预测该类，结合编码错误的问题，就会出现全0输出的情况。

修复方案 #

1. 修正目标变量编码 #

替换OneHotEncoder为LabelEncoder，代码如下：

# 读取训练数据时保留DataFrame结构，方便处理
df_train = pd.read_csv("train.csv")
X_train = df_train.iloc[:, 1:-1]
y_train = df_train['Fertilizer Name']

# 用LabelEncoder处理目标变量
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
y_train = le.fit_transform(y_train)

# 预测后转回原始肥料名称
y_pred = classifier.predict(X_test)
inverse_transformed_data = le.inverse_transform(y_pred)

2. 修正特征预处理流程 #

用列名而非索引处理特征，避免编码后索引失效的问题：

# 明确区分数值和分类特征列名
numeric_cols = ['Temparature', 'Humidity', 'Moisture', 'Nitrogen', 'Potassium', 'Phosphorous']
categorical_cols = ['Soil Type', 'Crop Type']

# 用ColumnTransformer一站式处理编码和标准化
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler

ct_x = ColumnTransformer(
    transformers=[
        ('onehot', OneHotEncoder(sparse_output=False), categorical_cols),
        ('scaler', StandardScaler(), numeric_cols)
    ],
    remainder='drop'
)
X_train = ct_x.fit_transform(X_train)
X_test = ct_x.transform(X_test)

3. 检查并处理类别不平衡 #

统计训练集类别分布：

print(df_train['Fertilizer Name'].value_counts(normalize=True))

若存在严重不平衡，可采取以下措施：

• 在模型中设置class_weight='balanced'，让模型自动调整类别权重：

classifier = RandomForestClassifier(n_estimators=500, max_depth=25, random_state=42, class_weight='balanced')

• 对少数类做过采样，或对多数类做欠采样。

4. 验证模型训练集表现 #

先在训练集小批量数据上测试，确认模型是否学到东西：

y_pred_train = classifier.predict(X_train[:100])
print(y_pred_train)
print(le.inverse_transform(y_pred_train))

若训练集表现正常，再排查测试集问题；若训练集也输出全0，需进一步检查数据预处理逻辑。

内容的提问来源于stack exchange，提问作者hacks_and_nimbus