MNIST数据集下神经网络不收敛准确率仅10%求技术协助

Hey there, sorry to hear you're stuck getting your neural network to converge—hitting ~10% accuracy on MNIST is exactly what random guessing gives you, so there's definitely a key piece missing in your implementation. Let's break down the most likely issues based on what you've shared and Tariq Rashid's book:

1. Fix That Obvious Typo First #

Looking at your code snippet, the line self.hid_layer = hid_la... is cut off, but it looks like a typo (you probably meant hid_layer instead of hid_la). If this variable is misnamed, every part of your network that relies on the hidden layer size will break—like weight initialization or forward propagation. Double-check all variable names for typos first; small mistakes like this can sink your whole model.

2. Weight Initialization is Critical #

Rashid's book emphasizes initializing weights to small, normally distributed values scaled by the square root of the number of input nodes to prevent sigmoid saturation. If you skipped this step (or used something like np.random.randint which generates large values), your sigmoid activation will get stuck at 0 or 1, making gradients vanish and the network unable to learn.

Your weight initialization should look something like this:

def __init__(self, inp_layer, hid_layer, out_layer, lr):
    self.inp_layer = inp_layer
    self.hid_layer = hid_layer
    self.out_layer = out_layer
    self.lr = lr  # Don't forget to define learning rate!
    
    # Initialize weights with normal distribution, scaled by node count
    self.wih = np.random.normal(0.0, pow(self.hid_layer, -0.5), (self.hid_layer, self.inp_layer))
    self.who = np.random.normal(0.0, pow(self.out_layer, -0.5), (self.out_layer, self.hid_layer))
    
    # Sigmoid activation function
    self.activation_function = lambda x: scipy.special.expit(x)

3. Double-Check Data Preprocessing #

MNIST's raw pixel values are 0-255, which is way too wide for the sigmoid function (it will saturate to 1, killing gradients). Rashid's book normalizes pixels to the range 0.01 to 0.99 (avoiding 0/1 to keep sigmoid responsive). You also need to convert labels to one-hot encoded vectors (e.g., label 7 becomes [0.01, 0.01, ..., 0.99, ..., 0.01]).

If you skipped either step, your network has no way to learn meaningful patterns. Here's the preprocessing snippet from the book:

# Scale inputs to 0.01-0.99
scaled_input = (np.asfarray(all_values[1:]) / 255.0 * 0.99) + 0.01

# Create one-hot target output
targets = np.zeros(out_layer) + 0.01
targets[int(all_values[0])] = 0.99

4. Validate Forward/Backward Propagation Logic #

Make sure you're correctly applying the activation function during forward pass, and using its derivative (sigmoid(x) * (1 - sigmoid(x))) during backpropagation. A common mistake is forgetting to multiply by the derivative when calculating weight updates—this means your gradients are wrong, so weights never update in a useful way.

Your backpropagation code should include lines like this:

def train(self, inputs_list, targets_list):
    # Convert lists to 2D arrays
    inputs = np.array(inputs_list, ndmin=2).T
    targets = np.array(targets_list, ndmin=2).T
    
    # Forward pass
    hidden_inputs = np.dot(self.wih, inputs)
    hidden_outputs = self.activation_function(hidden_inputs)
    
    final_inputs = np.dot(self.who, hidden_outputs)
    final_outputs = self.activation_function(final_inputs)
    
    # Calculate errors
    output_errors = targets - final_outputs
    hidden_errors = np.dot(self.who.T, output_errors)
    
    # Update weights
    self.who += self.lr * np.dot((output_errors * final_outputs * (1.0 - final_outputs)), np.transpose(hidden_outputs))
    self.wih += self.lr * np.dot((hidden_errors * hidden_outputs * (1.0 - hidden_outputs)), np.transpose(inputs))

5. Check Learning Rate and Training Iterations #

Rashid uses a learning rate of ~0.1 for this model. If your learning rate is way too high, weights will oscillate wildly and never converge; if it's too low, learning will be glacial (though 10% accuracy suggests it's not just slow). Also, make sure you're training for enough epochs—try at least 5-10 passes over the full dataset.

Quick Debug Tip #

Test with a tiny subset of MNIST (e.g., 100 images) and print the loss after each epoch. If the loss doesn't decrease at all, you've got a fundamental logic error (like weights not being updated, or inputs/targets being misaligned). If it decreases but plateaus early, you might need more epochs or a tweak to the learning rate.

If you can share your full forward/backward propagation and training loop code, we can pinpoint the issue even faster—but start with these checks; they cover 90% of cases where MNIST networks get stuck at random-guess accuracy.

内容的提问来源于stack exchange，提问作者Mudit Aggarwal