Python3复现Nielsen MNIST神经网络无法学习问题排查

• 尝试复现Michael Nielsen神经网络教程第一章的MNIST手写数字识别代码，原代码基于Python 2.7编写，本次使用Python 3实现。
• 网络可以正常完成样本遍历、权重与偏置更新流程，但完全没有学习效果，测试集准确率稳定在10%左右，和随机猜测水平相当。
• 此前直接复制原代码在Python 2.7环境运行可以达到95%左右的准确率。当前实现和原示例只有两处核心差异：
  • 使用自行从MNIST官网下载的最新数据集
  • 为了方便跟踪数组形状，统一使用(N,)格式一维数组代替原代码的(N,1)列向量，将反向传播中两处计算权重梯度的np.dot替换为np.outer，核对过各层矩阵运算维度匹配，学习率、网络层尺寸等超参数和原示例完全一致。
• 目前未定位到网络失效原因，求排查思路。
• 完整实现代码如下：

import matplotlib.pyplot as plt
import numpy as np
import idx2numpy
import random

### LOAD DATASET ###
train = idx2numpy.convert_from_file("mnist/train-images.idx3-ubyte")
train_labels = idx2numpy.convert_from_file("mnist/train-labels.idx1-ubyte")
test = idx2numpy.convert_from_file("mnist/t10k-images.idx3-ubyte")
test_labels = idx2numpy.convert_from_file("mnist/t10k-labels.idx1-ubyte")

def vectorize(x):
    e = np.zeros(10)
    e[x] = 1.0
    return e

training_images = [np.reshape(i, (784))/255 for i in train]
training_labels = [vectorize(i) for i in train_labels]
training_set = list(zip(training_images,training_labels))

test_images = [np.reshape(i, (784))/255 for i in test]
test_set = list(zip(training_images,test_labels)) # 错误点

### NETWORK CLASS ###
class myNet():
    def __init__ (self , sizes ):
        self.sizes = sizes
        self.N = len(sizes)
        self.w = [np.random.randn(y, x) for x, y in zip( sizes [: -1] , sizes [1:]) ]
        self.b = [np.random.randn(i) for i in sizes[1:]]

    def sigmoid (self,z):
        return 1.0/(1.0+ np.exp(-z))

    def sigmoid_prime (self,z):
        return self.sigmoid (z)*(1 - self.sigmoid (z))

    def cost_derivative (self,output_activations , y):
        return ( output_activations - y)

    def feedforward (self , a):
        for bb, ww in zip(self.b , self.w ):
            a = self.sigmoid (np.dot(ww, a)+bb)
        return a

    def backprop (self , x, y):
        nabla_b = [np. zeros (bb. shape ) for bb in self.b ]
        nabla_w = [np. zeros (ww. shape ) for ww in self.w ]
        activation = x
        activations = [x] 
        zs = [] 
        for bb, ww in zip(self.b , self. w ):
            z = np.dot(ww, activation )+bb
            zs. append (z)
            activation = self.sigmoid (z)
            activations . append ( activation )
        # 反向传播
        delta = self. cost_derivative ( activations [-1], y) * self.sigmoid_prime (zs [ -1])
        nabla_b [-1] = delta
        nabla_w [-1] = np.outer(delta , activations [ -2])

        for l in range (2, self.N ):
            z = zs[-l]
            sp = self.sigmoid_prime (z)
            delta = np.dot(self.w [-l+1]. transpose () , delta ) * sp
            nabla_b [-l] = delta
            nabla_w [-l] = np.outer(delta , activations [-l -1])
        return (nabla_b , nabla_w )

    def update(self,mini_batch,eta):
        nabla_b = [np.zeros (bb.shape ) for bb in self.b ]
        nabla_w = [np.zeros (ww.shape ) for ww in self.w ]
        for x, y in mini_batch :
            delta_nabla_b , delta_nabla_w = self. backprop (x, y)
            nabla_b = [nb+dnb for nb , dnb in zip(nabla_b , delta_nabla_b )]
            nabla_w = [nw+dnw for nw , dnw in zip(nabla_w , delta_nabla_w )]
        self.w = [ww -( eta/len( mini_batch ))*nw
                         for ww, nw in zip(self.w , nabla_w )]
        self.b = [bb -( eta/len( mini_batch ))*nb
                        for bb, nb in zip(self.b, nabla_b )]
        return

    def gradient_descent(self,training_data,epochs,mini_batch_size,eta,test_data):
        n = len( training_data )
        for j in range (epochs):
            random.shuffle (training_data)
            mini_batches = [
                training_data [k:k+ mini_batch_size ]
                for k in range (0, n, mini_batch_size )]
            for mini_batch in mini_batches :
                self.update( mini_batch , eta)
            print("Epoch {0}: {1}". format (
                j, self.evaluate(test_data)))
        return

    def evaluate (self, test_data):
        test_results = [( np.argmax (self.feedforward (x)), y)
                        for (x, y) in test_data ]
        return sum(int(x == y) for (x, y) in test_results )

sizes =[28*28, 30, 10]
net = myNet(sizes)
net.gradient_descent(training_set,30,10,3.0,test_set)

和你换Python3、改np.outer、新下载的数据集都没关系，就是个手滑的低级错误：构造测试集的时候把训练集图片和测试集标签配对了。
你写的错误行：

test_set = list(zip(training_images,test_labels))

前面明明已经预处理完存了test_images，这里错写成了training_images。Python的zip会按长度更短的列表截断，最后生成的1万条测试数据，全是训练集前1万张图搭配完全不相关的测试集标签，二者没有任何对应关系。不管你模型训练得有多准，拿训练图的预测结果去对完全无关的测试标签，准确率自然就是10%上下的随机猜测水平。

把这行里的training_images换成test_images即可：

test_set = list(zip(test_images,test_labels))

改完直接跑就能达到原代码95%左右的准确率。
补充：你把列向量换成一维数组、用np.outer计算权重梯度的写法完全正确，维度匹配没有问题，这部分不用调整。

内容的提问来源于stack exchange，提问作者nfko