如何计算灰度图像下简单深度网络参数及CNN权重数量

Let's walk through how to compute the number of trainable parameters for both your simple deep neural network (DNN) and a typical convolutional neural network (CNN) using the grayscale input shape you provided ((32, 32, 1)).

1. Simple Deep Neural Network (DNN) Parameters #

First, let's recap your input setup:

input_shape = (32, 32, 1)
flat_input_size = input_shape[0]*input_shape[1]*input_shape[2]  # 32*32*1 = 1024
num_classes = 4

Your DNN uses fully connected (Dense) layers, and each Dense layer's parameter count depends on:

• The number of input units to the layer
• The number of output units (specified by units)
• A bias term for each output unit (so add units to the weight count)

Step-by-Step Calculation for Your DNN Layers #

Your code defines these layers (we'll assume the final layer is Dense(units=num_classes) since num_classes=4):

dnn_model = Sequential()
dnn_model.add(Dense(input_dim=flat_input_size, units=1000))  # Layer 1
dnn_model.add(Activation("relu"))
dnn_model.add(Dense(units=512))  # Layer 2
dnn_model.add(Activation("relu"))
dnn_model.add(Dense(units=256))  # Layer 3
dnn_model.add(Activation("relu"))
dnn_model.add(Dense(units=num_classes))  # Final Layer (inferred)

Note: Activation layers have 0 trainable parameters—they only apply a non-linear transformation to the input without any weights to learn.

Now calculate each Dense layer's parameters:

• Layer 1 (Dense 1000):
  Parameters = (Input units × Output units) + Output units
  = (1024 × 1000) + 1000 = 1,024,000 + 1,000 = 1,025,000
• Layer 2 (Dense 512):
  Input units here are the output units of Layer 1 (1000)
  = (1000 × 512) + 512 = 512,000 + 512 = 512,512
• Layer 3 (Dense 256):
  Input units = 512
  = (512 × 256) + 256 = 131,072 + 256 = 131,328
• Final Layer (Dense 4):
  Input units = 256
  = (256 × 4) + 4 = 1,024 + 4 = 1,028

Total DNN Parameters #

Add all these together:
1,025,000 + 512,512 + 131,328 + 1,028 = 1,669,868 trainable parameters.

2. CNN Weight/Parameter Calculation #

For CNNs, most parameters come from convolutional (Conv2D) layers, plus any fully connected layers at the end. Let's use a typical CNN example with your grayscale input:

Key Formula for Conv2D Layers #

Each convolutional layer's parameters are calculated as:
(Kernel Height × Kernel Width × Input Channels) × Number of Filters + Number of Filters
The extra Number of Filters accounts for the bias term per filter.

Example CNN Setup #

Let's define a simple CNN for your input:

cnn_model = Sequential()
# Convolutional Layer 1
cnn_model.add(Conv2D(filters=32, kernel_size=(3,3), activation='relu', input_shape=(32,32,1)))
# Pooling Layer (no parameters)
cnn_model.add(MaxPooling2D(pool_size=(2,2)))
# Convolutional Layer 2
cnn_model.add(Conv2D(filters=64, kernel_size=(3,3), activation='relu'))
# Pooling Layer (no parameters)
cnn_model.add(MaxPooling2D(pool_size=(2,2)))
# Flatten and add Dense layers
cnn_model.add(Flatten())
cnn_model.add(Dense(units=128, activation='relu'))
cnn_model.add(Dense(units=num_classes, activation='softmax'))

Step-by-Step Parameter Calculation #

• Conv2D Layer 1 (32 filters, 3x3 kernel):
  Input channels = 1 (grayscale)
  = (3×3×1)×32 + 32 = 288 + 32 = 320
• Conv2D Layer 2 (64 filters, 3x3 kernel):
  Input channels here are the number of filters from Layer 1 (32)
  = (3×3×32)×64 + 64 = 288×64 + 64 = 18,432 + 64 = 18,496
• Pooling Layers: MaxPooling2D has 0 parameters—it just downsamples the feature maps without trainable weights.
• Flatten Layer: No parameters, just reshapes the 2D feature maps into a 1D vector.
• Dense Layer 1 (128 units):
  First, calculate the flattened size after pooling:
  After first pooling: (32-3+1) → 30 → MaxPool2D(2,2) → 15
  After second pooling: (15-3+1) →13 → MaxPool2D(2,2) →6 (integer division)
  Flattened size: 6×6×64 =2304
  Parameters: (2304×128)+128=294,912 +128= 295,040
• Final Dense Layer (4 units):
  =(128×4)+4=512+4= 516

Total CNN Parameters #

Add all together:320 +18,496 +295,040 +516= 314,372 trainable parameters.

Key Takeaways #

• For DNNs: All parameters are in fully connected layers (weights + biases per unit).
• For CNNs: Most parameters are in convolutional kernels (shared weights across the input space), plus any subsequent fully connected layers.
• Activation and pooling layers do not contribute trainable parameters.

内容的提问来源于stack exchange，提问作者HungryBird