从零实现Python神经网络反转8位二进制列表时输出固定，求助排查

为验证神经网络原理，从零实现了一个Python神经网络，任务是将8项0-1列表反转（例如输入[0, 1, 0, 1, 0, 1, 0, 1]应输出[1, 0, 1, 0, 1, 0, 1, 0]）。但训练完成后，无论输入何种数据，模型始终输出相同的预测值（通常为0.2689…和0.7310…），以下是代码：

#imports
import random as r
import numpy as np
import matplotlib.pyplot as plt 
import plotly.express as px
import pandas as pd
from tqdm.notebook import tqdm


#State Constants
nNeurons = 8 #Number of neurons per layer
nLayers = 24 #Number of layers
nWeights = nNeurons + 1
trainingDataSize = 10000 #Training data size
nIterations = 250 #Number of iterations of each algorithm
testSize = 100 #Number of data points used for a loss function for the whole network
nAttempts = 5 #Number of models gradient descent back propegation generates

#Data

#Objective function
def f(x):
    output = []
    for i in range(len(x)):
        output.append(x[-i-1])
    return(output)

#Random 8-bit binary vector
def randomX():
    output = []
    for i in range(8):
        output.append(r.choice([0,1]))
    return(output)

#Generating the training data
trainingData = []
for i in range(trainingDataSize):
    x = randomX()
    trainingData.append([x, f(x)])

#Functions
def sigmoid(x):
    if x < -10:
        return(0)
    else:
        return(1/(1 + np.e**(-x)))

def neuron(w, x):
    output = w[0]
    for i in range(len(x)):
        output += w[i + 1] * x[i]
    activation = tanh(output)
    return(sigmoid(activation))

def generateWeights():
    return(100 * (np.random.rand(nLayers, nNeurons, nWeights) - 0.5))

def hiddenLayer(layerNum, inp, hiddenWeights):
    w = hiddenWeights[layerNum]
    output = []
    for i in range(nNeurons):
        output.append(neuron(w[i],inp))
    return(output)

def neuralNetwork(inp, weights):
    lay = inp
    for i in range(nLayers):
        lay = hiddenLayer(i, lay, weights)
    return(lay)

#Loss of a single data point based on the current weights
def lossOne(weights):
    dataPoint = r.choice(trainingData)
    output = 0
    prediction = neuralNetwork(dataPoint[0], weights)
    actual = dataPoint[1]
    for i in range(len(prediction)):
        output += 0.5 * (prediction[i] - actual[i]) ** 2
    return(output)

#Loss of several data points
def loss(weights):
    output = 0
    for i in range(testSize):
        output += lossOne(weights)
    return(output)

def hiddenLayer(layerNum, inp, hiddenWeights):
w = hiddenWeights[layerNum]
output = []
for i in range(nNeurons):
    output.append(neuron(w[i],inp))
return(output)

def layerLoss(layerNum, inp, weights, expectedOut):
    output = 0
    actualOut = hiddenLayer(layerNum, inp, weights)
    for i in range(len(expectedOut)):
        output += 0.5 * (actualOut[i] - expectedOut[i]) ** 2
    return(output)

scale = 100
dw = 0.01
bestY = 10 ** 10 #For tracking which weight is best
for h in tqdm(range(nAttempts)):
    currentW = generateWeights()

    #Using back-propegation to train the network with gradient descent
    for i in tqdm(range(nIterations)):

        dataPoint = r.choice(trainingData)
        lay = dataPoint[0]
        expectedOut = dataPoint[1]

        for j in range(nLayers):

            for k in range(nNeurons):

                #Gradient descent on the kth neuron in layer - j - 1
                for l in range(j): #Finding the input this layer recieives
                    lay = hiddenLayer(l, lay, currentW)
            
                currentY = layerLoss(-j - 1, lay, currentW, expectedOut)

                nearbyW = currentW
                nearbyY = layerLoss(-j - 1, lay, nearbyW, expectedOut)

                dY = nearbyY - currentY
                pert = r.uniform(-dw ** 2,dw ** 2) #Small perturbation to prevent getting stuck at maxima

                currentW[-j - 1][k] -= (scale * dY/dw) + pert


            expectedOut = lay
        
        
    #Keeping the best model
    newY = loss(currentW)
    if newY < bestY:
        bestW = currentW
        bestY = newY

#Testing the neural network
tstx = [1, 1, 0, 0, 1, 1, 0, 0]
tstGD = neuralNetwork(tstx, bestW)

错误原因分析 #

1. 语法错误与函数重复定义 #

代码中重复定义了hiddenLayer函数，且第二次定义没有缩进，属于语法错误。这会导致后续调用hiddenLayer时使用的是有语法问题的版本，前向传播和反向传播的逻辑完全混乱，无法正确计算层输出。

2. 反向传播逻辑完全错误 #

• 梯度计算方式错误：你试图通过修改单一层的权重来计算损失变化，但layerLoss只计算该层输出与预期值的损失，而非整个网络最终损失对权重的梯度，这完全不符合反向传播的核心逻辑——需要从输出层往回逐层计算损失对每一层权重的梯度。
• 权重浅拷贝问题：nearbyW = currentW是浅拷贝，修改nearbyW会直接改变原currentW的值，导致currentY和nearbyY的计算基于同一组权重，梯度计算完全失效。
• 中间结果未缓存：反向传播需要前向传播过程中每一层的输入和输出值来计算梯度，但你在训练循环中反复修改lay变量，没有保存这些关键中间数据，根本无法正确推导梯度。
• 随机扰动破坏训练：添加的随机扰动pert会干扰梯度下降的方向，让模型无法沿着损失降低的方向更新权重，反而导致训练混乱。

3. 激活函数与权重初始化问题 #

• 未定义tanh函数：neuron函数中调用了tanh(output)，但代码中没有导入tanh（既没有from math import tanh也没有使用np.tanh），这会直接抛出NameError，导致激活函数逻辑异常，模型输出异常值。
• 权重初始化过大：generateWeights生成的权重范围是[-50,50]，如此大的初始权重会让sigmoid函数的输入绝对值过大，导致sigmoid输出接近0或1，进入饱和区，梯度趋近于0，模型无法更新权重，最终输出固定值。

4. 训练流程不合理 #

• 单样本随机训练：每次迭代只选一个随机样本，且没有正确计算该样本对所有权重的梯度，而是错误地逐层处理，导致权重更新完全偏离正确方向，模型无法学习到输入输出的映射关系。
• 损失计算不稳定：lossOne每次随机选取样本，loss函数累加100个随机样本的损失，训练时没有使用固定批次或合理的样本更新策略，模型的损失反馈不稳定，无法有效收敛。

5. 网络结构过于复杂 #

24层的深度对于“8位列表反转”这种简单任务来说完全没必要，过深的网络结合sigmoid激活函数会加剧梯度消失问题，模型根本无法训练。

内容的提问来源于stack exchange，提问作者Daisy Welham