PyTorch训练MNIST模型：Loss下降但Accuracy完全不变的问题排查

我是PyTorch新手，参照官方CIFAR10教程搭建了首个基于MNIST数据集的神经网络。训练时Loss显著下降，但准确率始终维持在0.09857142857142857完全不变。

训练代码 #

for epoch in range(5): # a epoch is one iteration over all the data
    
        running_loss = 0.0
        for i, data in enumerate(train_loader, 0):
            # get the inputs (this is a batch, so lists of images & labels)
            inputs, labels = data
            
            # reset the gradient
            optimizer.zero_grad()
            
            # forward + backward + optimize
            outputs = net(inputs) # forward
            loss = criterion(outputs, labels) # loss
            loss.backward() # backward
            optimizer.step() # update
            
            # print statistics
            running_loss += loss.item()
            if i % 2000 == 1999: # print every 2000 mini-batches
                print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 2000:.3f}')
                running_loss = 0.0
        
        # compute accuracy after each epoch
        correct = 0
        total = 0

        with torch.no_grad():
            for data in train_loader:
                images, labels = data
                output = net(images)
                _, predicted = torch.max(outputs.data, 1)
                total += labels.size(0)
                correct += (predicted == labels).sum().item()

        print(correct/total)

print('Training finished')

训练输出 #

[1,  2000] loss: 1.902
[1,  4000] loss: 1.413
[1,  6000] loss: 1.188
[1,  8000] loss: 0.844
[1, 10000] loss: 0.666
[1, 12000] loss: 0.631
[1, 14000] loss: 0.614
[1, 16000] loss: 0.534
[1, 18000] loss: 0.557
[1, 20000] loss: 0.518
[1, 22000] loss: 0.448
[1, 24000] loss: 0.450
[1, 26000] loss: 0.466
[1, 28000] loss: 0.493
[1, 30000] loss: 0.412
[1, 32000] loss: 0.404
0.09857142857142857
[2,  2000] loss: 0.442
[2,  4000] loss: 0.421
[2,  6000] loss: 0.429
[2,  8000] loss: 0.423
[2, 10000] loss: 0.411
[2, 12000] loss: 0.426
[2, 14000] loss: 0.425
[2, 16000] loss: 0.400
[2, 18000] loss: 0.430
[2, 20000] loss: 0.376
[2, 22000] loss: 0.385
[2, 24000] loss: 0.376
[2, 26000] loss: 0.388
[2, 28000] loss: 0.433
[2, 30000] loss: 0.346
[2, 32000] loss: 0.357
0.09857142857142857
[3,  2000] loss: 0.393
[3,  4000] loss: 0.356
[3,  6000] loss: 0.396
[3,  8000] loss: 0.381
[3, 10000] loss: 0.350
[3, 12000] loss: 0.368
[3, 14000] loss: 0.405
[3, 16000] loss: 0.355
[3, 18000] loss: 0.367
[3, 20000] loss: 0.355
[3, 22000] loss: 0.357
[3, 24000] loss: 0.366
[3, 26000] loss: 0.362
[3, 28000] loss: 0.393
[3, 30000] loss: 0.336
[3, 32000] loss: 0.333
0.09857142857142857
[4,  2000] loss: 0.372
[4,  4000] loss: 0.323
[4,  6000] loss: 0.362
[4,  8000] loss: 0.368
[4, 10000] loss: 0.346
[4, 12000] loss: 0.345
[4, 14000] loss: 0.381
[4, 16000] loss: 0.363
[4, 18000] loss: 0.357
[4, 20000] loss: 0.337
[4, 22000] loss: 0.363
[4, 24000] loss: 0.343
[4, 26000] loss: 0.353
[4, 28000] loss: 0.390
[4, 30000] loss: 0.298
[4, 32000] loss: 0.343
0.09857142857142857
[5,  2000] loss: 0.350
[5,  4000] loss: 0.324
[5,  6000] loss: 0.361
[5,  8000] loss: 0.350
[5, 10000] loss: 0.307
[5, 12000] loss: 0.347
[5, 14000] loss: 0.372
[5, 16000] loss: 0.347
[5, 18000] loss: 0.356
[5, 20000] loss: 0.302
[5, 22000] loss: 0.339
[5, 24000] loss: 0.345
[5, 26000] loss: 0.340
[5, 28000] loss: 0.405
[5, 30000] loss: 0.310
[5, 32000] loss: 0.352
0.09857142857142857
Training finished

问题原因 #

准确率计算代码里存在变量名错误：

_, predicted = torch.max(outputs.data, 1)

这里误用了训练循环中最后一个batch的outputs变量，而非当前batch的output。在整个准确率计算循环中，始终用同一个固定的输出结果做预测，导致预测结果完全不随当前batch的图像变化。MNIST有10个类别，随机猜测的准确率约为0.1，和你看到的0.09857数值吻合。

解决方法 #

将准确率计算中的outputs.data替换为当前batch的output.data：

_, predicted = torch.max(output.data, 1)

内容的提问来源于stack exchange，提问作者a_hayler