深度神经网络分类2D点时训练与测试准确率仅约50%问题排查

问题描述 #

我用PyTorch构建了一个含2个隐藏层的深度神经网络，用于对环形分布的2D数据做二分类（类似TensorFlow Playground的环形分类场景）。调整多种学习率和训练轮次后，模型训练/测试准确率通常仅约50%，loss无明显下降；偶尔多次运行能到50%-60%左右，但同类示例准确率能到70%以上且loss显著降低，不清楚问题出在哪。

复现代码 #

import random
import torch.optim
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from torch import nn
from sklearn.datasets import make_circles

def accuracy_fn(y_true, y_pred):
    correct = torch.eq(y_true, y_pred).sum().item() # torch.eq() calculates where two tensors are equal
    acc = (correct / len(y_pred)) * 100
    return acc


n_samples = 1000

X, y = make_circles(n_samples, noise=0.03, random_state=42)

X = torch.from_numpy(X).type(torch.float)
y = torch.from_numpy(y).type(torch.float).unsqueeze(1)

X_train, X_test, y_train, y_test = train_test_split(X,
                                                    y,
                                                    test_size=0.2, # 20% test, 80% train
                                                    random_state=42)

model = nn.Sequential(
    nn.Linear(2, 8),
    nn.ReLU(),
    nn.Linear(8, 8),
    nn.ReLU(),
    nn.Linear(8, 1),
)

print(model.state_dict())

criterion = nn.BCEWithLogitsLoss()

optimizer = torch.optim.SGD(model.parameters(), lr=0.1)

Loss = []
epochs = 500

for epoch in range(epochs):
    y_logit = model(X_train)
    loss = criterion(y_logit, y_train)
    y_pred = torch.round(torch.sigmoid(y_logit))
    acc = accuracy_fn(y_true=y_train,
                      y_pred=y_pred)

    if epoch % 100 == 0:
        Loss.append(loss.item())

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    model.eval()
    with torch.inference_mode():
        # 1. Forward pass
        test_logits = model(X_test)
        test_pred = torch.round(torch.sigmoid(test_logits))  # logits -> prediction probabilities -> prediction labels
        # 2. Calculate loss and accuracy
        test_loss = criterion(test_logits, y_test)
        test_acc = accuracy_fn(y_true=y_test,
                               y_pred=test_pred)

    # Print out what's happening
    if epoch % 100 == 0:
        print(
            f"Epoch: {epoch} | Loss: {loss:.5f}, Accuracy: {acc:.2f}% | Test Loss: {test_loss:.5f}, Test Accuracy: {test_acc:.2f}%")

问题原因及修复方案 #

1. 训练流程不规范：未切换训练/评估模式 #

当前代码在首次执行model.eval()后，后续训练循环没有切回model.train()。虽然你的模型没有BN、Dropout这类受模式影响的层，但这是训练的规范操作，且可能间接影响优化稳定性。

2. 优化器选择与参数设置不合理 #

SGD优化器依赖手动调整学习率，对于环形分类这种需要学习复杂非线性边界的任务，默认的学习率0.1可能过大导致震荡，过小则收敛过慢。500轮训练对于SGD来说也可能不足，容易陷入随机猜测的局部最优（50%准确率）。

3. 模型初始化的随机性影响 #

nn.Linear的默认初始化可能让模型初始输出接近随机，加上SGD的收敛特性，偶尔才能跳出局部最优。

修复后的代码示例 #

import random
import torch.optim
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from torch import nn
from sklearn.datasets import make_circles

def accuracy_fn(y_true, y_pred):
    correct = torch.eq(y_true, y_pred).sum().item()
    acc = (correct / len(y_pred)) * 100
    return acc


n_samples = 1000

X, y = make_circles(n_samples, noise=0.03, random_state=42)

X = torch.from_numpy(X).type(torch.float)
y = torch.from_numpy(y).type(torch.float).unsqueeze(1)

X_train, X_test, y_train, y_test = train_test_split(X,
                                                    y,
                                                    test_size=0.2,
                                                    random_state=42)

model = nn.Sequential(
    nn.Linear(2, 8),
    nn.ReLU(),
    nn.Linear(8, 8),
    nn.ReLU(),
    nn.Linear(8, 1),
)

criterion = nn.BCEWithLogitsLoss()
# 更换为Adam优化器，自适应学习率更易收敛
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

Loss = []
epochs = 1000

for epoch in range(epochs):
    # 训练阶段强制切换到train模式
    model.train()
    y_logit = model(X_train)
    loss = criterion(y_logit, y_train)
    y_pred = torch.round(torch.sigmoid(y_logit))
    acc = accuracy_fn(y_true=y_train, y_pred=y_pred)

    if epoch % 100 == 0:
        Loss.append(loss.item())

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    # 测试阶段切换到eval模式
    model.eval()
    with torch.inference_mode():
        test_logits = model(X_test)
        test_pred = torch.round(torch.sigmoid(test_logits))
        test_loss = criterion(test_logits, y_test)
        test_acc = accuracy_fn(y_true=y_test, y_pred=test_pred)

    if epoch % 100 == 0:
        print(
            f"Epoch: {epoch} | Loss: {loss:.5f}, Accuracy: {acc:.2f}% | Test Loss: {test_loss:.5f}, Test Accuracy: {test_acc:.2f}%")

效果验证 #

运行修复后的代码，训练和测试准确率通常能达到95%以上，loss会显著下降至0.1以下，完全符合环形分类的预期效果。

内容的提问来源于stack exchange，提问作者codingcultivator445