多项式回归阶数低于4时输出直线，无法拟合2/3阶模型问题咨询

问题背景 #

需要实现三阶多项式回归，无数据分析相关经验，套用现有代码代入自有数据时出现异常：y数据已完成标准化，x原始值为1950-2018年的年份序列。

异常表现 #

• 直接使用年份作为x值时完全无法正常拟合，无论选择多少阶数，输出结果都是直线
• 改用索引值作为x输入拟合时，仅4阶及以上的模型可以正常输出曲线，2阶、3阶多项式拟合效果不符合预期

原代码 #

x = np.array(list(range(1, 70)))
y = np.array([-1.07312323, -1.12360264, -1.16848888, -1.21237286, -1.24931163,
   -1.24563078, -1.25029589, -1.25804974, -1.26992981, -1.2759396 ,
   -1.31707672, -1.28845207, -1.2553561 , -1.21670196, -1.17405228,
   -1.13823657, -1.10201293, -1.0652651 , -1.01830663, -0.95872599,
   -0.86864519, -0.77287454, -0.67380868, -0.56936508, -0.47234488,
   -0.38025164, -0.28073984, -0.17953134, -0.08026437,  0.01376177,
    0.09177617,  0.15270399,  0.2005737 ,  0.23841612,  0.2860362 ,
    0.34606907,  0.39385415,  0.44154466,  0.49050035,  0.5338063 ,
    0.58003198,  0.61416929,  0.59416923,  0.56887929,  0.53366038,
    0.4907952 ,  0.45338928,  0.40975728,  0.35098762,  0.29307093,
    0.24168722,  0.21576624,  0.25267974,  0.3066606 ,  0.37672389,
    0.45321951,  0.53410345,  0.62491894,  0.72720349,  0.81841313,
    0.9213128 ,  1.03645707,  1.15479503,  1.25998302,  1.35221566,
    1.44653627,  1.52833712,  1.60458778,  1.68225894])

# transforming the data to include another axis
x = x[:, np.newaxis]
y = y[:, np.newaxis]

polynomial_features= PolynomialFeatures(degree=4)
x_poly = polynomial_features.fit_transform(x)

model = LinearRegression()
model.fit(x_poly, y)
y_poly_pred = model.predict(x_poly)

plt.scatter(x, y, s=10)
plt.plot(x, y_poly_pred, color='m')
plt.show()

拟合效果示例 #

问题原因 #

异常由x未做特征缩放导致：你使用的1~69索引或原始年份数值偏大，计算多项式高阶项时数值会指数级增长，不同阶特征的数值差距可达几个数量级，线性回归求解时低阶项的系数会被数值精度淹没，最终拟合效果和纯线性回归一致。4阶及以上能输出曲线是因为你的数据存在明显拐点，高阶项的权重足够大才能抵消精度误差显现出曲线效果。

修复方案 #

在生成多项式特征前先对x做标准化/归一化，把x的数值缩放到相近量级即可，修改后的代码如下：

import numpy as np
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt

x = np.array(list(range(1, 70)))
y = np.array([-1.07312323, -1.12360264, -1.16848888, -1.21237286, -1.24931163,
   -1.24563078, -1.25029589, -1.25804974, -1.26992981, -1.2759396 ,
   -1.31707672, -1.28845207, -1.2553561 , -1.21670196, -1.17405228,
   -1.13823657, -1.10201293, -1.0652651 , -1.01830663, -0.95872599,
   -0.86864519, -0.77287454, -0.67380868, -0.56936508, -0.47234488,
   -0.38025164, -0.28073984, -0.17953134, -0.08026437,  0.01376177,
    0.09177617,  0.15270399,  0.2005737 ,  0.23841612,  0.2860362 ,
    0.34606907,  0.39385415,  0.44154466,  0.49050035,  0.5338063 ,
    0.58003198,  0.61416929,  0.59416923,  0.56887929,  0.53366038,
    0.4907952 ,  0.45338928,  0.40975728,  0.35098762,  0.29307093,
    0.24168722,  0.21576624,  0.25267974,  0.3066606 ,  0.37672389,
    0.45321951,  0.53410345,  0.62491894,  0.72720349,  0.81841313,
    0.9213128 ,  1.03645707,  1.15479503,  1.25998302,  1.35221566,
    1.44653627,  1.52833712,  1.60458778,  1.68225894])

# 新增x标准化步骤
scaler = StandardScaler()
x_scaled = scaler.fit_transform(x[:, np.newaxis])
y = y[:, np.newaxis]

# 可自由设置degree为2、3、4
polynomial_features = PolynomialFeatures(degree=3)
x_poly = polynomial_features.fit_transform(x_scaled)

model = LinearRegression()
model.fit(x_poly, y)
y_poly_pred = model.predict(x_poly)

plt.scatter(x, y, s=10)
plt.plot(x, y_poly_pred, color='m')
plt.show()

修改后即可正常输出2、3阶的多项式拟合曲线。

内容的提问来源于stack exchange，提问作者Whatever