实现线性回归SGD算法时输出NaN问题求助

1. 核心问题：未对特征做标准化处理 #

你代码中导入了StandardScaler但未实际应用。随机梯度下降（SGD）对特征尺度极度敏感，当不同特征的数值量级差异过大时（比如Population是数千级，而Urban是0/1），梯度会出现爆炸式增长，权重很快变为无穷大（inf），最终演变为NaN。

解决方法：在划分特征与标签后，对特征进行标准化：

scaler = StandardScaler()
train_data_X = scaler.fit_transform(train_data_X)

注意：标准化后train_data_X会转为numpy数组，后续代码中需将train_data_X.iloc[i]改为train_data_X[i]（数组没有iloc方法）。

2. 次要问题：截距（intercept）维度不匹配 #

初始时intercept是标量0.1，但第一次计算y_estimated = np.dot(x_i, w_matrix) + intercept时，np.dot(x_i, w_matrix)输出是形状为(1,)的数组，导致y_estimated也是数组，进而grad_intercept成为数组。更新后intercept变为数组，后续计算会出现维度广播错误，加速数值异常。

解决方法：将预测值转为标量，保证intercept始终为标量：

y_estimated = np.dot(x_i, w_matrix).item() + intercept

3. 额外优化建议 #

• 洗牌数据：SGD每次迭代应打乱数据顺序，避免模型陷入局部最优或收敛缓慢。在每个epoch开始时添加洗牌逻辑：

  for e in range(epoch_num):
      shuffled_indices = np.random.permutation(len(train_data_X))
      X_shuffled = train_data_X[shuffled_indices]
      Y_shuffled = train_data_Y.iloc[shuffled_indices].values
      for i in range(len(X_shuffled)):
          x_i = X_shuffled[i]
          y_i = Y_shuffled[i]
          # 后续计算逻辑不变
  

• 调整学习率：配合标准化，可尝试更小的学习率（比如0.001），避免权重更新幅度过大。

修正后的完整代码 #

import pandas as pd
from sklearn.preprocessing import StandardScaler
import numpy as np

# 数据预处理
train_data = pd.read_csv('carseats_train.csv')
train_data.replace({'Yes': 1, 'No': 0}, inplace=True)
onehot_tr = pd.get_dummies(train_data['ShelveLoc'], dtype=int, prefix_sep='_', prefix='ShelveLoc')
train_data = train_data.drop('ShelveLoc', axis=1)
train_data = train_data.join(onehot_tr)

train_data_Y = train_data.iloc[:, 0]
train_data_X = train_data.drop('Sales', axis=1)

# 特征标准化
scaler = StandardScaler()
train_data_X = scaler.fit_transform(train_data_X)

# SGD实现
learning_rate = 0.001
epoch_num = 50
initial_w = 0.1
intercept = 0.1
w_matrix = np.ones((12, 1)) * initial_w

for e in range(epoch_num):
    # 每个epoch洗牌数据
    shuffled_indices = np.random.permutation(len(train_data_X))
    X_shuffled = train_data_X[shuffled_indices]
    Y_shuffled = train_data_Y.iloc[shuffled_indices].values
    
    for i in range(len(X_shuffled)):
        x_i = X_shuffled[i]
        y_i = Y_shuffled[i]
        
        # 确保预测值为标量
        y_estimated = np.dot(x_i, w_matrix).item() + intercept
        
        grad_w = x_i.reshape(-1, 1) * (y_i - y_estimated)
        grad_intercept = (y_i - y_estimated)
        
        w_matrix = w_matrix - 2 * learning_rate * grad_w
        intercept = intercept - 2 * learning_rate * grad_intercept

print("Final weights:\n", w_matrix)
print("Final intercept:", intercept)

内容的提问来源于stack exchange，提问作者Merora