逻辑回归中使用梯度下降替换fmin_tnc从零搭建模型的相关问题

你原有代码已经完成了sigmoid函数、前向计算、代价函数、梯度计算的基础逻辑，只需要修改fit函数替换掉第三方优化方法即可，梯度下降的实现逻辑如下： #

修改点说明 #

• 新增梯度下降超参数：学习率alpha、最大迭代次数，也可按需加入代价变化阈值作为提前终止条件
• 迭代过程中每次用梯度乘以学习率更新theta参数，直到达到终止条件

完整可运行代码 #

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import random

data = pd.read_csv("train.txt", header=None)
testing_data = pd.read_csv("test.txt")

# X = 特征值，除最后一列外的所有列
X = data[data.columns[:-1]]

# y = 标签值，数据框的最后一列
y = data.iloc[:, -1]

X = np.c_[np.ones((X.shape[0], 1)), X]
y = y[:, np.newaxis]
theta = np.zeros((X.shape[1], 1))

def sigmoid(x):
    return (1 / (1 + np.exp(-x)))

def net_input(theta, x):
    return np.dot(x, theta)

def probability(theta, x):
    return sigmoid(net_input(theta, x))

def cost_function(theta, x, y):
    m = x.shape[0]
    total_cost = -(1 / m) * np.sum(
        y * np.log(probability(theta, x)) + (1 - y) * np.log(
            1 - probability(theta, x)))
    return total_cost

def gradient(theta, x, y):
    # 计算theta点对应的代价函数梯度
    m = x.shape[0]
    return (1 / m) * np.dot(x.T, sigmoid(net_input(theta, x)) - y)

def fit(x, y, theta, alpha=0.01, epochs=100000):
    m = x.shape[0]
    cost_history = []
    for i in range(epochs):
        grad = gradient(theta, x, y)
        theta = theta - alpha * grad
        # 可选保存每轮代价，用来检查收敛情况
        cost = cost_function(theta, x, y)
        cost_history.append(cost)
        # 可选：每1000轮打印一次代价，方便观察训练进度
        if i % 1000 == 0:
            print(f"迭代轮次 {i}, 代价值: {cost:.4f}")
    return theta.flatten(), cost_history

parameters, cost_history = fit(X, y, theta)

def predict(x):
    theta = parameters[:, np.newaxis]
    return probability(theta, x)

def accuracy(x, actual_classes, probab_threshold=0.5):
    predicted_classes = (predict(x) >= 
                         probab_threshold).astype(int)
    predicted_classes = predicted_classes.flatten()
    accuracy = np.mean(predicted_classes == actual_classes)
    return accuracy * 100

print(f"训练集准确率: {accuracy(X, y.flatten()):.2f}%")

# 可选：绘制代价收敛曲线
plt.rcParams["font.sans-serif"] = ["SimHei"]
plt.plot(cost_history)
plt.xlabel("迭代次数")
plt.ylabel("代价值")
plt.title("梯度下降收敛曲线")
plt.show()

调参说明 #

• 学习率alpha可以根据收敛情况调整：如果代价震荡不下降，说明学习率过大，需要调小；如果代价下降非常缓慢，说明学习率过小，可以调大
• 迭代次数可以根据代价收敛情况调整，当代价基本不再下降时即可停止迭代

内容的提问来源于stack exchange，提问作者Sankhojyoti Halder