MNIST数字识别PyTorch模型准确率停滞9.84%，求故障排查

我参考YouTube教程的NumPy实现搭建MNIST数字识别模型，改用PyTorch重构后，训练5个epoch准确率始终卡在9.84%，完全没有提升，输出如下：

Acc: 9.84%
Acc: 9.84%
Acc: 9.84%
Acc: 9.84%
Acc: 9.84%

我的PyTorch代码 #

import torch
import torch.nn.functional as F
import pandas as pd

"""
w = weights, b = bias, i = input, h = hidden, o = output, l = label
e.g. w_i_h = weights from input layer to hidden layer
"""
"""
w = weights, b = bias, i = input, h = hidden, o = output, l = label
e.g. w_i_h = weights from input layer to hidden layer
"""

# Read the CSV file into a pandas DataFrame, considering the first row as column labels
data = pd.read_csv('train.csv', header=0)

# Convert the label and pixel columns to a PyTorch tensor
labels = torch.tensor(data['label'].values)
labels = labels.view(-1, 1)
labels = F.one_hot(labels.squeeze(), num_classes=10) # onehot the labels
images = torch.tensor(data.iloc[:, 1:].values)

train_images = images[:, :40000] 
test_images = images[:, 40000:42000]

# divide the rgb coding to probabilities
# 1 = black, 0 = white
train_images = train_images.float() / 255
test_images = test_images.float() / 255

# w_i_h = weights from input layer to hidden layer
w_i_h = torch.rand(10, 784) - 0.5

# b_i_h = biases from input layer to hidden layer
b_i_h = torch.zeros(10, 1)

# w_h_o = weights from hidden layer to output layer
w_h_o = torch.rand(10, 10) - 0.5

# b_h_o = biases from hidden layer to output layer
b_h_o = torch.zeros(10, 1)  

epochs = 1
learn_rate = 0.01

epochs = 1
learn_rate = 0.1

for epoch in range(epochs):
    nr_correct = 0
    for i, l in zip(train_images, labels):
        # make then image from column of 784 to matrix of size (784, 1)
        i = i.view(784, 1)
        l = l.view(10, 1)
        
        # forward propagation
        # input layer -> hidden layer        
        z1 = w_i_h @ i + b_i_h
        relu = torch.nn.ReLU() # because you can't call it directly
        a1 = relu(z1)

        # hidden layer -> output layer
        z2 = w_h_o @ a1 + b_h_o
        a2 = F.softmax(z2, dim=1) # dim=0 specifies that we soft max along columns / from top to bottom

        # error / cost calculation
        cost = 1 / len(l) * torch.sum((a2 - l) ** 2)

        # check if the biggest output prediction matches the correct label
        nr_correct += int(torch.argmax(a2) == torch.argmax(l))
        
        # backward propagation
        # output layer -> hidden layer
        o_err = a2 - l
        w_h_o = -learn_rate * o_err @ a1.T
        b_h_o = -learn_rate * o_err

        # hidden layer -> input layer
        h_err = w_h_o.mm(o_err) * (a1 > 0)
        w_i_h = -learn_rate * h_err @ i.T
        b_i_h = -learn_rate * h_err

    # Show accuracy for this epoch
    print(f"Acc: {round((nr_correct / len(labels)) * 100, 2)}%")

教程的NumPy代码 #

from data import get_mnist
import numpy as np
import matplotlib.pyplot as plt


"""
w = weights, b = bias, i = input, h = hidden, o = output, l = label
e.g. w_i_h = weights from input layer to hidden layer
"""
images, labels = get_mnist()
w_i_h = np.random.uniform(-0.5, 0.5, (20, 784))
w_h_o = np.random.uniform(-0.5, 0.5, (10, 20))
b_i_h = np.zeros((20, 1))
b_h_o = np.zeros((10, 1))

learn_rate = 0.01
nr_correct = 0
epochs = 1
for epoch in range(epochs): # 5
    for img, l in zip(images, labels):
        img.shape += (1,)
        l.shape += (1,)

        # Forward propagation input -> hidden
        h_pre = b_i_h + w_i_h @ img
        h = 1 / (1 + np.exp(-h_pre))
        print(img)

        # Forward propagation hidden -> output
        o_pre = b_h_o + w_h_o @ h
        o = 1 / (1 + np.exp(-o_pre))

        # Cost / Error calculation
        e = 1 / len(o) * np.sum((o - l) ** 2, axis=0)
        nr_correct += int(np.argmax(o) == np.argmax(l))
        
        # Backpropagation output -> hidden (cost function derivative)
        delta_o = o - l
        w_h_o += -learn_rate * delta_o @ np.transpose(h)
        b_h_o += -learn_rate * delta_o
        # Backpropagation hidden -> input (activation function derivative)
        delta_h = np.transpose(w_h_o) @ delta_o * (h * (1 - h))
        w_i_h += -learn_rate * delta_h @ np.transpose(img)
        b_i_h += -learn_rate * delta_h
        break
    # Show accuracy for this epoch
    print(f"Acc: {round((nr_correct / images.shape[0]) * 100, 2)}%")
    nr_correct = 0

核心错误点与修正方案 #

1. 权重/偏置更新方式完全错误 #

原NumPy代码中，权重更新是累加梯度调整（基于原有权重做微调）：

w_h_o += -learn_rate * delta_o @ np.transpose(h)

而你的PyTorch代码是直接赋值替换：

w_h_o = -learn_rate * o_err @ a1.T

这相当于每次迭代都把权重完全重置为当前梯度的负值，完全丢弃之前的权重信息，模型根本无法学习任何特征，只能保持随机猜测水平。

修正：把所有权重/偏置的赋值=改成累加+=。

2. 数据切分维度完全颠倒 #

你的代码中：

train_images = images[:, :40000] 

images的维度是(样本数, 784像素)，[:, :40000]会取每个样本的前40000个像素（MNIST只有784像素，实际是取全部像素，但样本数没切分），正确的切分应该是取前40000个样本：

train_images = images[:40000, :] 
test_images = images[40000:42000, :]
train_labels = labels[:40000]

否则训练用的样本和标签完全不匹配，模型无法学习。

3. 隐藏层神经元数量不匹配原模型 #

原NumPy代码的隐藏层是20个神经元：

w_i_h = np.random.uniform(-0.5, 0.5, (20, 784))

而你的PyTorch代码写成了10个：

w_i_h = torch.rand(10, 784) - 0.5

这导致隐藏层和输出层维度一致，失去特征提取作用，同时梯度计算逻辑混乱。

修正：对齐原模型的隐藏层维度：

w_i_h = torch.rand(20, 784) - 0.5
b_i_h = torch.zeros(20, 1)
w_h_o = torch.rand(10, 20) - 0.5

4. 输出层激活函数与原模型不一致 #

原NumPy代码用sigmoid作为输出激活，你的代码用了softmax，虽然softmax适合分类，但和MSE损失搭配时梯度效率极低，且和原模型逻辑不一致。

修正：替换为sigmoid：

a2 = torch.sigmoid(z2)

5. 准确率计算分母错误 #

你用len(labels)作为准确率分母，但训练时只遍历了train_images（40000个样本），应该用len(train_images)：

print(f"Acc: {round((nr_correct / len(train_images)) * 100, 2)}%")

修正后的完整PyTorch代码 #

import torch
import torch.nn.functional as F
import pandas as pd

# 读取数据
data = pd.read_csv('train.csv', header=0)

# 处理标签和图像
labels = torch.tensor(data['label'].values)
labels = F.one_hot(labels, num_classes=10).float()
images = torch.tensor(data.iloc[:, 1:].values).float() / 255

# 正确切分训练/测试集
train_images = images[:40000, :]
train_labels = labels[:40000]
test_images = images[40000:42000, :]
test_labels = labels[40000:42000]

# 初始化权重和偏置，对齐原模型维度
w_i_h = torch.rand(20, 784) - 0.5
b_i_h = torch.zeros(20, 1)
w_h_o = torch.rand(10, 20) - 0.5
b_h_o = torch.zeros(10, 1)

epochs = 5
learn_rate = 0.01

for epoch in range(epochs):
    nr_correct = 0
    for i, l in zip(train_images, train_labels):
        # 调整维度
        i = i.view(784, 1)
        l = l.view(10, 1)
        
        # 前向传播
        z1 = w_i_h @ i + b_i_h
        a1 = F.relu(z1)

        z2 = w_h_o @ a1 + b_h_o
        a2 = torch.sigmoid(z2)

        # 统计正确数
        nr_correct += int(torch.argmax(a2) == torch.argmax(l))
        
        # 反向传播（累加更新）
        o_err = a2 - l
        w_h_o += -learn_rate * o_err @ a1.T
        b_h_o += -learn_rate * o_err

        h_err = w_h_o.T @ o_err * (a1 > 0).float()
        w_i_h += -learn_rate * h_err @ i.T
        b_i_h += -learn_rate * h_err

    # 打印训练准确率
    print(f"Epoch {epoch+1} Acc: {round((nr_correct / len(train_images)) * 100, 2)}%")

内容的提问来源于stack exchange，提问作者josill