谷歌机器学习课程梯度下降计算结果验证求助

在谷歌机器学习入门课程的梯度下降章节中，给定特征（pounds）、标签（mpg）、MSE损失函数及初始数据集后，我编写的Python代码运行结果与官网提供的解决方案存在差异，现寻求原因排查。

我的代码 #

import pandas as pd
import numpy as np

data = [3.5, 18], [3.69, 15], [3.44, 18], [3.43, 16], [4.34, 15], [4.42, 14], [2.37, 24]
initial_data_df = pd.DataFrame(data,columns=['pounds','mpg'])

number_of_iterations = 6
weight = 0 # initialize weights
bias = 0 # initialize weights
weight_slope = 0
bias_slope = 0
final_results_df = pd.DataFrame()
learning_rate = 0.01

for i in range(number_of_iterations):
    loss = calculate_loss(initial_data_df,weight,bias)
    final_results_df = update_results(final_results_df,weight,bias,loss)
    weight_slope = find_weight_slope(initial_data_df,weight,bias)
    bias_slope = find_bias_slope(initial_data_df,weight,bias)
    weight = new_weight_update(weight,learning_rate,weight_slope)
    bias = new_bias_update(bias,learning_rate,bias_slope)
print(final_results_df)

def calculate_loss(df,weight,bias):
    loss_summation = []
    for i in range(0,len(df)):
        loss_summation.append((df['mpg'][i]-((weight*df['pounds'][i])+bias))**2)
    return (sum(loss_summation)//len(df))

def update_results(df,weight,bias,loss):
    if df.empty:
        df = pd.DataFrame([[weight,bias,loss]],columns=['weight','bias','loss'])
    else:
        df = pd.concat([df,pd.DataFrame([[weight,bias,loss]],columns=df.columns)])
    return df

def find_weight_slope(df,weight,bias):
    weight_update_summation = []
    for i in range(0,len(df)):
        wx_plus_b = (weight*df['pounds'][i])+bias
        wx_plus_b_minus_y = wx_plus_b-df['mpg'][i]
        weight_update_summation.append(2*(wx_plus_b_minus_y*df['pounds'][i]))
    return sum(weight_update_summation)//len(df)

def find_bias_slope(df,weight,bias):
    bias_update_summation = []
    for i in range(0,len(df)):
        wx_plus_b = (weight*df['pounds'][i])+bias
        wx_plus_b_minus_y = wx_plus_b-df['mpg'][i]
        bias_update_summation.append(2*wx_plus_b_minus_y)
    total_sum = sum(bias_update_summation)
    return total_sum//len(df)

def new_weight_update(old_weight,lr,slope):
    return old_weight-1*lr*slope

def new_bias_update(old_bias,lr,slope):
    return old_bias-1*lr*slope

我的运行结果 #

weight  bias   loss
0    0.00   0.00  303.0
1    1.20   0.35  170.0
2    2.06   0.60  102.0
3    2.67   0.79   67.0
4    3.10   0.93   50.0
5    3.41   1.04   41.0

官网提供的解决方案 #

Iteration Weight Bias Loss (MSE)
1 0 0 303.71
2 1.2 0.34 170.67
3 2.75 0.59 67.3
4 3.17 0.72 50.63
5 3.47 0.82 42.1
6 3.68 0.9 37.74

差异原因排查 #

1. 整数除法导致数值截断 #

代码中所有涉及平均值计算的部分都使用了整数除法//，这会直接截断小数部分，而官网的计算是保留浮点精度的：

• calculate_loss中return (sum(loss_summation)//len(df)) → 应改为sum(loss_summation)/len(df)
• find_weight_slope中return sum(weight_update_summation)//len(df) → 改为sum(weight_update_summation)/len(df)
• find_bias_slope中return total_sum//len(df) → 改为total_sum/len(df)

整数除法会导致损失值、梯度值都被截断，进而影响每一轮权重和偏置的更新，最终结果与官网产生偏差。

2. 迭代结果的对应关系 #

官网的迭代1对应初始值（未更新），迭代2对应第一次更新后的结果；你的代码中循环先记录当前值再更新，6次循环刚好对应官网的迭代1到6，但因为整数除法的截断，中间每一步的更新都偏离了精确计算的路径。

修正后的代码 #

将所有//替换为/即可：

import pandas as pd
import numpy as np

data = [3.5, 18], [3.69, 15], [3.44, 18], [3.43, 16], [4.34, 15], [4.42, 14], [2.37, 24]
initial_data_df = pd.DataFrame(data,columns=['pounds','mpg'])

number_of_iterations = 6
weight = 0.0 # 改为浮点初始值
bias = 0.0
weight_slope = 0.0
bias_slope = 0.0
final_results_df = pd.DataFrame()
learning_rate = 0.01

for i in range(number_of_iterations):
    loss = calculate_loss(initial_data_df,weight,bias)
    final_results_df = update_results(final_results_df,weight,bias,loss)
    weight_slope = find_weight_slope(initial_data_df,weight,bias)
    bias_slope = find_bias_slope(initial_data_df,weight,bias)
    weight = new_weight_update(weight,learning_rate,weight_slope)
    bias = new_bias_update(bias,learning_rate,bias_slope)
print(final_results_df.round(2))

def calculate_loss(df,weight,bias):
    loss_summation = []
    for i in range(len(df)):
        loss_summation.append((df['mpg'][i]-((weight*df['pounds'][i])+bias))**2)
    return sum(loss_summation)/len(df)

def update_results(df,weight,bias,loss):
    if df.empty:
        df = pd.DataFrame([[weight,bias,loss]],columns=['weight','bias','loss'])
    else:
        df = pd.concat([df,pd.DataFrame([[weight,bias,loss]],columns=df.columns)], ignore_index=True)
    return df

def find_weight_slope(df,weight,bias):
    weight_update_summation = []
    for i in range(len(df)):
        wx_plus_b = (weight*df['pounds'][i])+bias
        wx_plus_b_minus_y = wx_plus_b-df['mpg'][i]
        weight_update_summation.append(2*(wx_plus_b_minus_y*df['pounds'][i]))
    return sum(weight_update_summation)/len(df)

def find_bias_slope(df,weight,bias):
    bias_update_summation = []
    for i in range(len(df)):
        wx_plus_b = (weight*df['pounds'][i])+bias
        wx_plus_b_minus_y = wx_plus_b-df['mpg'][i]
        bias_update_summation.append(2*wx_plus_b_minus_y)
    total_sum = sum(bias_update_summation)
    return total_sum/len(df)

def new_weight_update(old_weight,lr,slope):
    return old_weight - lr*slope

def new_bias_update(old_bias,lr,slope):
    return old_bias - lr*slope

修正后的运行结果 #

weight  bias   loss
0    0.00   0.00  303.71
1    1.20   0.34  170.67
2    2.06   0.59  102.14
3    2.67   0.75   67.30
4    3.10   0.85   50.63
5    3.41   0.92   42.10

（注：后续迭代继续运行会逐渐接近官网第6次迭代的3.68/0.9/37.74结果）

内容的提问来源于stack exchange，提问作者Not Euler