如何从遍历多alpha的Lasso回归中提取最低MSE对应的最优alpha及变量指数

核心问题说明 #

• 原代码遍历alpha的逻辑无效：循环所有alpha后仅保留了最后一个alpha的拟合结果，未完成alpha参数优选
• 原存储结构仅记录了特征组合对应的MSE，未同步存储最优alpha、多项式特征映射等必要信息

调整后完整代码 #

import numpy as np
from itertools import combinations
from sklearn.linear_model import LassoCV
from sklearn.preprocessing import PolynomialFeatures
from sklearn.model_selection import cross_validate

x_combos = []
feature_pool = ['Date', 'Cargo_size', 'Parcel_size', 'Rest', 'Age', 'Sub', 'X_coord', 'Y_coord']
for n in range(1,9):
    combos = combinations(feature_pool, n)
    x_combos.extend(combos)

lasso_models = {}
alphas = 10**np.linspace(10,-2, 100)*.5    

for n in range(0, len(x_combos)):
    combo_list = list(x_combos[n])
    x = data[combo_list]
    poly = PolynomialFeatures(3, include_bias=False)
    poly_x = poly.fit_transform(x)
    # 用LassoCV自动完成alpha交叉验证优选
    model = LassoCV(alphas=alphas, max_iter=100000, normalize=True, cv=10)
    model.fit(poly_x, y)
    # 计算交叉验证MSE
    cv_scores = cross_validate(model, poly_x, y, cv=10, scoring='neg_mean_squared_error', return_train_score=True)
    avg_mse = abs(np.mean(cv_scores['test_neg_mean_squared_error']))
    # 存储所有必要信息
    lasso_models[str(combo_list)] = {
        'avg_mse': avg_mse,
        'best_alpha': model.alpha_,
        'poly_transformer': poly,
        'best_model': model
    }

# 查找最优结果
print("最佳Lasso回归模型输出：")
# 找最小MSE对应的特征组合
best_combo_str = min(lasso_models.keys(), key=lambda k: lasso_models[k]['avg_mse'])
best_result = lasso_models[best_combo_str]
print("最低平均测试MSE:", best_result['avg_mse'].round(2))
print("对应变量组合：", best_combo_str)
# 输出最优alpha
print("对应最优alpha参数：", best_result['best_alpha'])

# 输出变量对应指数和系数
print("\n变量指数与对应系数：")
poly_feature_names = best_result['poly_transformer'].get_feature_names_out(input_features=eval(best_combo_str))
for feat_name, coef in zip(poly_feature_names, best_result['best_model'].coef_):
    # 过滤系数为0的无效特征
    if abs(coef) > 1e-6:
        print(f"特征表达式 {feat_name}， 系数：{coef.round(4)}")

功能说明 #

• 最优alpha提取：使用LassoCV替代手动循环alpha，训练完成后直接通过model.alpha_属性获取交叉验证选出的最优alpha值
• 变量对应指数获取：通过PolynomialFeatures的get_feature_names_out()方法可以直接输出每个多项式特征的表达式，比如Cargo_size^2、Cargo_size Parcel_size这类形式，能清晰看到每个特征对应的变量和指数

内容的提问来源于stack exchange，提问作者Cody Olivotto