线性回归梯度下降运行异常：RuntimeWarning及成本值过大求助

问题描述 #

每次运行线性回归的梯度下降算法时，都会出现3条RuntimeWarning，且计算得到的cost值异常巨大，需要技术帮助。

原代码 #

import matplotlib.pyplot as plt
import pandas as pd

data = pd.read_csv(r"C:\Users\antho\OneDrive\Dokumenty\Data\warsawflatprices.csv.csv", delimiter=",")
data = pd.DataFrame(data)
x_train = data["SizeM2"]
y_train = data["Price"]
m = len(x_train)

def compute_cost(x, y, w, b):
    cost = 0
    for i in range(m):
        cost += (w * x[i] + b - y[i])**2
    cost = cost / (2 * m)
    return cost

def compute_gradient(x, y, w, b):
    dj_dw = 0
    dj_db = 0
    for i in range(m):
        dj_dw += (w * x[i] + b - y[i]) * x[i]
        dj_db += (w * x[i] + b - y[i])
    dj_dw = dj_dw / m
    dj_db = dj_db / m
    return dj_dw, dj_db

def gradient_descent(x, y, w, b, alpha, num_iters):
    cost_history = []
    w_history = []
    b_history = []
    for i in range(num_iters):
        dj_dw, dj_db = compute_gradient(x, y, w, b)
        w = w - alpha * dj_dw
        b = b - alpha * dj_db
        cost = compute_cost(x, y, w, b)
        cost_history.append(cost)
        w_history.append(w)
        b_history.append(b)
    ind = cost_history.index(min(cost_history))
    cost_min = min(cost_history)
    w_min = w_history[ind]
    b_min = b_history[ind]
    print(f"The minimal cost is {cost_min}, the respected parameters are w = {w_min} and b = {b_min}")
gradient_descent(x_train, y_train, 0, 0, 0.01, 10000)

报错信息 #

C:\Users\antho\PycharmProjects\MachineLearningSpecialization\Warsaw Flat Prices (Linear Regression).py:13: RuntimeWarning: overflow encountered in scalar power
  cost += (w * x[i] + b - y[i])**2
C:\Users\antho\PycharmProjects\MachineLearningSpecialization\Warsaw Flat Prices (Linear Regression).py:21: RuntimeWarning: overflow encountered in scalar multiply
  dj_dw += (w * x[i] + b - y[i]) * x[i]
C:\Users\antho\PycharmProjects\MachineLearningSpecialization\Warsaw Flat Prices (Linear Regression).py:33: RuntimeWarning: invalid value encountered in scalar subtract
  w = w - alpha * dj_dw
The minimal cost is 365223891369846.8

解决方案 #

1. 调小学习率α #

你用的α=0.01太大了。房价类数据的特征（如面积）和目标值（价格）数值量级都不小，大学习率会让参数w、b在迭代中急剧变化，很快超出浮点数范围，直接引发溢出警告。
建议先把α调到1e-6或1e-5，再逐步测试找到合适的数值。

2. 对特征做缩放处理 #

特征SizeM2（几十到几百）和目标Price（几万到几十万）的数值量级差异极大，会导致梯度更新的步幅严重失衡，进一步加剧梯度爆炸。
解决方法是对特征做标准化（Z-score归一化），公式为：x_scaled = (x - x均值) / x标准差，这样能把特征缩放到均值为0、标准差为1的范围，让梯度更新更稳定。

3. 代码优化（可选但推荐） #

原代码用循环遍历每个样本，效率低且容易出错。改用numpy的向量运算替代循环，不仅计算更快，还能避免索引相关的问题。另外，没必要存储所有迭代的w、b和cost，只需跟踪当前最小cost对应的参数即可，节省内存。

修改后的示例代码 #

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

data = pd.read_csv(r"C:\Users\antho\OneDrive\Dokumenty\Data\warsawflatprices.csv.csv", delimiter=",")
x_train = data["SizeM2"].values
y_train = data["Price"].values
m = len(x_train)

# 特征标准化处理
x_mean = np.mean(x_train)
x_std = np.std(x_train)
x_train_scaled = (x_train - x_mean) / x_std

def compute_cost(x, y, w, b):
    # 向量运算替代循环，提升效率
    predictions = w * x + b
    cost = np.sum((predictions - y)**2) / (2 * m)
    return cost

def compute_gradient(x, y, w, b):
    predictions = w * x + b
    dj_dw = np.sum((predictions - y) * x) / m
    dj_db = np.sum(predictions - y) / m
    return dj_dw, dj_db

def gradient_descent(x, y, w, b, alpha, num_iters):
    cost_min = float('inf')
    w_min = w
    b_min = b
    for i in range(num_iters):
        dj_dw, dj_db = compute_gradient(x, y, w, b)
        w = w - alpha * dj_dw
        b = b - alpha * dj_db
        cost = compute_cost(x, y, w, b)
        # 实时跟踪最小cost对应的参数
        if cost < cost_min:
            cost_min = cost
            w_min = w
            b_min = b
    print(f"最小cost值为 {cost_min}，对应的参数w = {w_min}，b = {b_min}")
    # 将缩放后的参数还原到原始特征尺度
    w_original = w_min / x_std
    b_original = b_min - w_min * x_mean / x_std
    print(f"还原到原始特征尺度的参数：w = {w_original}，b = {b_original}")

# 使用缩放后的特征+合适的学习率训练
gradient_descent(x_train_scaled, y_train, 0, 0, 1e-3, 10000)

内容的提问来源于stack exchange，提问作者user22227348