从零实现神经网络Dense层前向传播报矩阵维度不匹配错误

报错本质是矩阵维度约定不一致+多层逻辑缺陷，具体问题点：

• 维度匹配逻辑完全混乱：输入批次形状为(batch_size, 特征数)（比如batch_size=10时单批输入形状是(10,784)），但Dense层权重初始化为(神经元数, 输入特征数)，前向传播时用np.dot(self.weights.T, inputs)计算，此时weights.T形状是(784,32)，和(10,784)的输入做矩阵乘法时，前者列数32不等于后者行数10，直接触发维度不对齐错误。
• 层间输入形状无自动推断：非首层Dense层没有传入inputs参数时，默认初始化为0，权重形状异常，就算第一层维度修对，后续层也会报错。
• 模型结构不符合任务要求：MNIST是10分类任务，原代码最后一层设为1个神经元接Softmax，输出维度为1，和10维的分类标签完全不匹配；且标签没有做one-hot编码，和输出维度无法对齐。
• 反向传播逻辑错误：反向传播时参数传递错误（把优化器配置对象当成学习率传入）、梯度计算维度不匹配、激活函数导数调用逻辑错误。
• Dropout实现逻辑错误：默认dropout_rate=1等价于输出全置0，且训练阶段没有随机掩码逻辑，仅做常数缩放，完全不符合Dropout的正则化逻辑。
• 输入未做归一化：原始MNIST像素值范围为0-255，直接输入会导致梯度爆炸/收敛极慢。

1. 统一矩阵维度约定 #

所有张量统一采用「样本维度在前，特征维度在后」的存储格式：

• 输入批次形状：(batch_size, n_input_features)
• Dense层权重形状：(n_input_features, n_neurons)
• 偏置形状：(1, n_neurons)
• 前向传播计算改为np.dot(inputs, self.weights) + self.biases，输出形状自动为(batch_size, n_neurons)，天然对齐。

2. 增加层自动构建逻辑 #

在NeuralNetwork初始化时遍历所有层，首层校验输入维度配置，后续层自动从前一层的神经元数推断输入维度，初始化对应权重，不需要手动给非首层传inputs参数。

3. 修正模型结构与数据处理 #

• MNIST10分类任务最后一层改为10个神经元接Softmax激活，删除多余的1神经元Dense层
• 对标签做one-hot编码，匹配最后一层的输出维度
• 输入像素值归一化到0-1区间，提升训练稳定性

4. 修正反向传播与Dropout逻辑 #

• 反向传播时正确传入学习率，梯度计算按统一维度调整矩阵乘法顺序，按batch大小做梯度平均
• 给激活函数、损失函数增加导数判断分支，反向传播时传入导数标记计算梯度，Softmax配合交叉熵使用简化梯度降低计算量
• Dropout默认失活率设为0（即不启用），训练阶段按失活率随机生成掩码、做反向缩放，推理阶段跳过Dropout逻辑。

import time
import numpy as np
from tensorflow.keras.datasets import mnist

# 激活函数实现，新增导数计算分支
class Activations:
    @staticmethod
    def ReLU(x, derivative=False):
        if derivative:
            return (x > 0).astype(float)
        return np.maximum(0, x)
    
    @staticmethod
    def Softmax(x, derivative=False):
        exps = np.exp(x - np.max(x, axis=1, keepdims=True)) # 防溢出
        res = exps / np.sum(exps, axis=1, keepdims=True)
        if derivative:
            return np.ones_like(res) # 配合交叉熵简化梯度
        return res

# 损失函数实现，新增导数计算分支
class Losses:
    @staticmethod
    def Categorical_Cross_Entropy(y_pred, y_true, derivative=False):
        y_pred = np.clip(y_pred, 1e-7, 1-1e-7) # 防log(0)
        if derivative:
            return y_pred - y_true # 配合Softmax的简化梯度
        return -np.sum(y_true * np.log(y_pred), axis=1, keepdims=True)

class Optimizers:
    SGD = 0

class Layers:
    class Dense:
        def __init__(self, neurons, activation=Activations.ReLU, inputs=None, dropout_rate=0):
            self.neurons = neurons
            self.n_inputs = inputs
            self.activation = activation
            self.dropout_rate = dropout_rate
            self.weights = None
            self.biases = None
        
        def build(self, n_inputs):
            # 权重延后到build阶段初始化，支持自动推断输入维度
            self.n_inputs = n_inputs
            self.weights = np.random.randn(n_inputs, self.neurons) * 0.01 # 小值初始化防梯度爆炸
            self.biases = np.zeros((1, self.neurons)) # 偏置初始化为0更稳定
        
        def forward(self, inputs, training=True):
            self.inputs = inputs
            self.linear_output = np.dot(inputs, self.weights) + self.biases
            self.outputs = self.activation(self.linear_output)
            # 仅训练阶段启用Dropout
            if training and self.dropout_rate > 0:
                self.dropout_mask = np.random.binomial(1, 1-self.dropout_rate, size=self.outputs.shape) / (1-self.dropout_rate)
                self.outputs *= self.dropout_mask
            return self.outputs
        
        def backward(self, error, learning_rate):
            if self.dropout_rate > 0:
                error *= self.dropout_mask
            # 激活层反向传播
            delta = error * self.activation(self.linear_output, derivative=True)
            # 计算梯度，按batch大小平均
            batch_size = self.inputs.shape[0]
            dW = np.dot(self.inputs.T, delta) / batch_size
            db = np.sum(delta, axis=0, keepdims=True) / batch_size
            # 传递到上一层的误差
            prev_error = np.dot(delta, self.weights.T)
            # 更新权重
            self.weights -= learning_rate * dW
            self.biases -= learning_rate * db
            return prev_error


class NeuralNetwork:
    def __init__(self, layers, loss, optimizer, learning_rate=0.01):
        self.layers = layers
        self.loss = loss
        self.optimizer = optimizer
        self.lr = learning_rate
        # 自动构建各层，推断输入维度
        prev_neurons = None
        for layer in self.layers:
            if layer.n_inputs is None:
                if prev_neurons is None:
                    raise ValueError("首层必须指定输入维度")
                layer.build(prev_neurons)
            else:
                layer.build(layer.n_inputs)
            prev_neurons = layer.neurons
    
    def forward(self, inputs, training=True):
        self.outputs = inputs
        for layer in self.layers:
            self.outputs = layer.forward(self.outputs, training=training)
        return self.outputs
    
    def backward(self, targets):
        delta = self.loss(self.outputs, targets, derivative=True)
        for layer in reversed(self.layers):
            delta = layer.backward(delta, self.lr)
    
    def train(self, inputs, targets, epochs=1, batch_size=1, verbose=False):
        self.epoch_losses = []
        self.epoch_accuracies = []
        total_start = time.time()
        for epoch in range(epochs):
            epoch_loss = 0
            correct = 0
            # 每轮打乱数据
            shuffle_idx = np.random.permutation(inputs.shape[0])
            x_shuf = inputs[shuffle_idx]
            y_shuf = targets[shuffle_idx]
            for i in range(0, inputs.shape[0], batch_size):
                batch_x = x_shuf[i:i+batch_size]
                batch_y = y_shuf[i:i+batch_size]
                pred = self.forward(batch_x, training=True)
                self.backward(batch_y)
                # 统计指标
                epoch_loss += np.sum(self.loss(pred, batch_y))
                correct += np.sum(np.argmax(pred, axis=1) == np.argmax(batch_y, axis=1))
            avg_loss = epoch_loss / inputs.shape[0]
            acc = correct / inputs.shape[0]
            self.epoch_losses.append(avg_loss)
            self.epoch_accuracies.append(acc)
            if verbose:
                print(f'Epoch {epoch+1}/{epochs} | Loss: {avg_loss:.4f} | Acc: {acc:.4f} | Time: {time.time()-total_start:.1f}s')
    
    def evaluate(self, inputs, targets):
        pred = self.forward(inputs, training=False)
        loss = np.mean(self.loss(pred, targets))
        acc = np.mean(np.argmax(pred, axis=1) == np.argmax(targets, axis=1))
        print(f'Test Result | Loss: {loss:.4f} | Acc: {acc:.4f}')
        return loss, acc

# 数据加载与预处理
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train = X_train.reshape(X_train.shape[0], -1) / 255.0
X_test = X_test.reshape(X_test.shape[0], -1) / 255.0
# 标签转one-hot编码
y_train_oh = np.zeros((y_train.shape[0], 10))
y_train_oh[np.arange(y_train.shape[0]), y_train] = 1
y_test_oh = np.zeros((y_test.shape[0], 10))
y_test_oh[np.arange(y_test.shape[0]), y_test] = 1

# 构建正确结构的模型
model = NeuralNetwork([
    Layers.Dense(32, Activations.ReLU, inputs=X_train.shape[1]),
    Layers.Dense(10, Activations.Softmax)
], Losses.Categorical_Cross_Entropy, Optimizers.SGD, learning_rate=0.1)

# 启动训练与测试
model.train(X_train, y_train_oh, epochs=20, batch_size=32, verbose=True)
model.evaluate(X_test, y_test_oh)

内容的提问来源于stack exchange，提问作者Daniel S.