提升ExtraTree回归模型性能的方法探讨

开发者阿泽

2026-7-4

如何构建性能优良的ExtraTree回归模型？

我希望基于指定数据集构建回归模型，尝试多种方法消除数据集中的异常值对模型性能的影响但未成功，扩大模型参数范围时会出现过拟合现象。请问如何构建性能优良的ExtraTree回归模型？

原始代码

url = 'https://raw.githubusercontent.com/ramazanunlu/RegressionModel/main/final_data_Jinit%20-%20Kopya.csv'
df = pd.read_csv(url,sep=";")

MMR=df[['C3','C4', 'C5', 'C6']]
MP=df[['C7', 'C8']]
OMP=df[['C9', 'C10']]
DIF=df[['Age (years)', 'C1', 'C2', 'Male', 'Female','Target']]

MAE=[]
MSE=[]
RMSE=[]
 
results=pd.DataFrame()

for i in range(len(MMR.columns)):
    data=pd.concat([MMR[MMR.columns[i]],MP,OMP,DIF],axis=1)
    data.head()
    
    #X=data.drop([' Kinit'],axis=1)
    X=data.drop(['Target'],axis=1)
    y=data['Target']

    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)
    columns=X_train.columns
    
    sc = StandardScaler()
    X_train[X.columns[0:8]] = sc.fit_transform(X_train[X.columns[0:8]])
    X_test[X.columns[0:8]] = sc.transform (X_test[X.columns[0:8]])
 
    n_estimators = [int(x) for x in np.linspace(start = 100, stop = 600, num = 6)]
    criterion=["squared_error", "absolute_error", "friedman_mse", "poisson"]
    min_samples_split = [2, 5, 10,12]
    min_samples_leaf = [2,4,6,12]
    max_depth = [5,10,15,20]
    max_features = ['sqrt','log2']

    random_grid = {
                   'n_estimators': n_estimators,
                  'criterion':criterion,
                   'max_features': max_features,
                   'max_depth': max_depth,
                   'min_samples_split': min_samples_split,
                   'min_samples_leaf': min_samples_leaf}

    rf = ExtraTreesRegressor()
    rf_random = RandomizedSearchCV(estimator = rf, param_distributions = random_grid,scoring='neg_mean_squared_error', n_iter = 50, cv = 5, verbose=1, random_state=42, refit=True)
    
    rf_random.fit(X_train,y_train)
      
    predictions=rf_random.predict(X_test)
    
    #predictions = boxcox_transformer_target.inverse_transform(predictions1.reshape(-1, 1))
    
    MAE.append(metrics.mean_absolute_error(y_test, predictions))
    MSE.append(metrics.mean_squared_error(y_test, predictions))
    RMSE.append(np.sqrt(metrics.mean_squared_error(y_test, predictions)))
    
    
    results=pd.concat([results,pd.DataFrame(predictions)],axis=1,ignore_index=True)

解决思路与代码调整建议

1. 异常值处理优化

替换鲁棒缩放器：StandardScaler对异常值敏感，换成RobustScaler，基于中位数和四分位数计算缩放规则，能有效降低异常值的干扰：

from sklearn.preprocessing import RobustScaler
sc = RobustScaler()
X_train[X.columns[0:8]] = sc.fit_transform(X_train[X.columns[0:8]])
X_test[X.columns[0:8]] = sc.transform(X_test[X.columns[0:8]])

精准定位异常值：用箱线图或Isolation Forest识别异常值，对目标变量Target的异常值尝试截断处理（比如将超过上四分位1.5倍四分位距的数值替换为上边界值），而非直接删除样本，避免数据量损失。

2. 抑制过拟合的参数调优

限制模型复杂度：缩小max_depth范围（比如3-10），同时增大min_samples_split和min_samples_leaf的取值，强制模型学习更通用的模式：
```
min_samples_split = [10, 15, 20]
min_samples_leaf = [5, 8, 10]
max_depth = [3, 5, 7, 10]
```
加入剪枝正则化：在参数网格中加入ccp_alpha，通过代价复杂度剪枝去掉冗余分支：
```
ccp_alpha = [0.001, 0.01, 0.05]
random_grid['ccp_alpha'] = ccp_alpha
```
调整特征采样：尝试max_features='auto'（使用所有特征）或更小的采样比例，减少单棵树的随机性，降低过拟合风险。

3. 训练流程优化

加入早停机制：用EarlyStoppingCV监控验证集误差，当误差不再下降时停止训练，避免过拟合：

from sklearn.experimental import enable_early_stopping
from sklearn.model_selection import EarlyStoppingCV

rf = ExtraTreesRegressor()
early_stopping = EarlyStoppingCV(estimator=rf, cv=5, scoring='neg_mean_squared_error', verbose=1, n_iter_no_change=5)
early_stopping.fit(X_train, y_train)
best_rf = early_stopping.best_estimator_

特征筛选：利用ExtraTrees的特征重要性，保留高贡献特征，减少噪声：

# 训练基础模型获取重要性
base_rf = ExtraTreesRegressor(random_state=42)
base_rf.fit(X_train, y_train)
# 筛选重要性前70%的特征
importances = pd.Series(base_rf.feature_importances_, index=X_train.columns)
selected_features = importances[importances >= importances.quantile(0.3)].index
X_train = X_train[selected_features]
X_test = X_test[selected_features]

4. 评估与验证优化

使用分层交叉验证：对于回归任务，可将Target分箱后用StratifiedKFold，确保训练集和测试集的目标分布一致：

from sklearn.model_selection import StratifiedKFold
# 对Target分箱
y_binned = pd.qcut(y_train, q=5, labels=False)
rf_random = RandomizedSearchCV(estimator=rf, param_distributions=random_grid, 
                               scoring='neg_mean_squared_error', n_iter=50, 
                               cv=StratifiedKFold(n_splits=5, shuffle=True, random_state=42), 
                               verbose=1, random_state=42, refit=True)

内容的提问来源于stack exchange，提问作者Ram

提升ExtraTree回归模型性能的方法探讨

原始代码 #

解决思路与代码调整建议 #

1. 异常值处理优化 #

2. 抑制过拟合的参数调优 #

3. 训练流程优化 #