首个卷积层反向传播失败：广播维度不匹配（stride=2异常）

为深入理解CNN原理，我未借助任何深度学习框架从零实现了一个卷积神经网络。网络的前向传播与反向传播流程大部分能正常执行，但在反向传播到第一个卷积层时出现维度不匹配问题。

网络结构 #

network = [
    Conv(input_shape=(128,96,96,3), kernel_shape=(4,4), num_kernels=40, stride=2, optimizer=GD_M, conv_mode="valid", kernel_initializer="he_normal", name="C1"),
    ReLU(name="R1"),
    ValidMaxPooling(3,2, name="MP1"),
    Conv(input_shape=(128,23,23,40), kernel_shape=(2,2), num_kernels=105, stride=1, optimizer=GD_M, conv_mode="valid", kernel_initializer="he_normal", name="C2"),
    ReLU(name="R2"),
    ValidMaxPooling(4,2, name="MP2"),
    Conv(input_shape=(128,10,10,105), kernel_shape=(2,2), num_kernels=158, stride=1, optimizer=GD_M, conv_mode="valid", kernel_initializer="he_normal", name="C3"),
    ReLU(name="R3"),
    Conv(input_shape=(128,9,9,158), kernel_shape=(2,2), num_kernels=158, stride=1, optimizer=GD_M, conv_mode="valid", kernel_initializer="he_normal", name="C4"),
    ReLU(name="R4"),
    Conv(input_shape=(128,8,8,158), kernel_shape=(2,2), num_kernels=105, stride=1, optimizer=GD_M, conv_mode="valid", kernel_initializer="he_normal", name="C5"),
    ReLU(name="R5"),
    ValidMaxPooling(3,2, name="MP3"),
    Flatten(name="F1"),
    Dense(input_neurons=945, output_neurons=945, optimizer=GD_M, name="D1"),
    ReLU(name="R6"),
    Dense(input_neurons=945, output_neurons=475, optimizer=GD_M, name="D2"),
    ReLU(name="R7"),
    Dense(input_neurons=475, output_neurons=8, optimizer=GD_M, name="D3"),
    Softmax(name="S1"),
]

问题细节 #

网络输入为128批次的96×96 RGB图像，第一层Conv是唯一设置stride=2的层。尽管该层的输入梯度无需传递给前层，但更新其权重与偏置时出现特征图/输入维度不匹配问题，且仅该层出现此问题。

Conv层反向传播代码 #

def backward(self, de_dy):
    de_db = de_dy
    de_dk_store = np.zeros(shape=(len(self.input),*self.kernels.shape)) 
    de_dx_store = np.zeros(shape=self.input.shape) 
    for b in range(self.batch_size): 
        for k in range(self.num_kernels): 
            for i in range(self.kernels.shape[1]): 
                de_dk_store[b,k,i] = correlate2d(self.input[b, i], de_dy[k], "valid")
                de_dx_store[b,i]+= convolve2d(de_dy[k], self.kernels[k,i], "full")
    de_dk_avg = np.mean(de_dk_store, axis=0)
    de_dx_avg = np.mean(de_dx_store, axis=0)
    self.kernels = self.optimizer.apply_optimizer(self.name+":K", self.kernels, de_dk_avg)
    self.biases = self.optimizer.apply_optimizer(self.name+":B", self.biases, de_db)
    return de_dx_avg

错误信息 #

de_dk_store[b,k,i] = correlate2d(self.input[b, i], de_dy[k], "valid")
ValueError: could not broadcast input array from shape (50,50) into shape (4,4)

具体来说，correlate2d(self.input[b, i], de_dy[k], "valid")的输出形状为(50,50)，无法赋值到形状为(4,4)的de_dk_store[b,k,i]位置。推测问题根源在于该层设置的stride=2，因为其他stride=1的层未出现此问题。

该层前向传播代码 #

def forward(self, input_array): # batch, channels, height, width
    self.input = input_array
    self.batch_size, self.channels, input_height, input_width = input_array.shape
    self.output_height = (input_height - self.pool_size) // self.stride + 1
    self.output_width = (input_width - self.pool_size) // self.stride + 1
    pooled_array = np.zeros((self.batch_size, self.channels, self.output_height, self.output_width))
    self.gradient_indexes = np.zeros(shape=(self.batch_size*self.channels*self.output_height*self.output_width, 4), dtype=int)
    z = 0
    for b in range(self.batch_size):
        for c in range(self.channels):    
            m = input_array[b][c]
            for i in range(self.output_height): # rows
                for j in range(self.output_width): # columns
                    patch = m[i*self.stride:i*self.stride+self.pool_size, j*self.stride:j*self.stride+self.pool_size]
                    pooled_array[b,c,i, j] = np.max(patch)
                    max_index_in_patch = np.unravel_index(np.argmax(patch, axis=None), patch.shape)
                    max_index_in_input = [b, c, int(i*self.stride) + int(max_index_in_patch[0]), int(j*self.stride) + int(max_index_in_patch[1])]
                    self.gradient_indexes[z] = max_index_in_input
                    z+=1                 
    return pooled_array

内容的提问来源于stack exchange，提问作者Bryan Carty