TensorFlow中CNN训练收敛至零向量，新手手部关节检测模型故障求助

Hey there! Building hand pose detection models from scratch can feel daunting when you're new to deep learning—let's walk through the problems you're facing and break down actionable fixes.

First: Let's Address Your Network Structure #

Your current setup is pretty minimal, which is likely a big reason the model isn't performing well. Here's where you can tweak it:

• Too few convolutional layers: A single 5x5, 8-channel conv layer on 240x320 depth images won't extract enough hierarchical features for hand joint detection. Hand poses need fine-grained features (like finger edges, knuckle shapes) that deeper conv stacks capture better. Try adding 1-2 more conv layers, increasing channel counts gradually (e.g., 8 → 16 → 32) and using smaller 3x3 kernels instead of 5x5—smaller kernels reduce computation and stack to capture complex features effectively.
• Mismatch in fully connected layer dimensions: Let's do a quick math check: after 2x2 max pooling, your feature map should be (240/2) x (320/2) x 8 = 120x160x8 = 153,600 parameters. But your first FC layer uses [38400, 1024]—that's a 75% reduction without any intermediate processing. Did you accidentally flatten only part of the feature map, or miss a channel-reduction step (like a 1x1 conv)? This mismatch can confuse the model during training and hurt performance.
• No regularization layers: Jumping straight from pooling to a large FC layer invites overfitting, and without dropout or batch normalization, training can be unstable. Add Dropout(0.5) after your FC layers, and batch normalization after each conv layer to stabilize gradients and improve generalization.

Next: Fixing the "Converging to Zero Vector" Problem #

This is a common issue in regression tasks (like joint coordinate prediction). Here are the most likely culprits:

• Unnormalized data: If your depth image values are in raw ranges (e.g., 0-255 or higher) and your joint coordinates are in pixel space (0-240/320), the loss values can be enormous. The model might learn that outputting zero is the easiest way to minimize loss quickly. Fix this by:
  • Normalizing depth images to the [0, 1] range (divide by the maximum depth value in your dataset)
  • Scaling joint coordinates to [0, 1] (divide x by 320, y by 240)
• Poor weight initialization: TensorFlow's default initializers work for simple models, but for deeper networks or large FC layers, they might lead to vanishing/exploding gradients. Switch to He initialization (great for ReLU activations) for conv and FC layers to keep gradients stable.
• Dead neurons from ReLU: If you're using ReLU activations, some neurons might "die" (output zero forever) if gradients are too small. Replace ReLU with LeakyReLU (with a small alpha like 0.1) to keep neurons active and gradients flowing.
• Learning rate is too high: A high learning rate can cause the model to overshoot optimal weights and collapse to a trivial solution (like all zeros). Start with a small learning rate (e.g., 1e-4) using the Adam optimizer, and add learning rate decay (e.g., reduce by 10% every 10 epochs) if training plateaus.
• Small batch size: Tiny batches lead to noisy gradient updates, making the model unstable. Try increasing your batch size to 32 or 64 if your GPU allows it—this smooths out gradient updates and helps the model converge properly.

Bonus: Dataset & Training Tips #

• Data augmentation: The ICVL dataset is useful but might not have enough variation to prevent overfitting. Add simple augmentations tailored to hand data:
  • Random cropping (e.g., crop to 224x224 to focus on the hand)
  • Random rotation (±15 degrees—make sure to rotate joint coordinates too!)
  • Horizontal flipping (flip the image and mirror joint x-coordinates)
• Check your loss function: For joint regression, MSE is standard, but make sure you're computing it correctly. If you're predicting multiple joints, ensure the loss averages over all joints instead of summing (summing can amplify loss and lead to unstable training).

Example Revised Network Structure #

Here's a more robust setup to try:

# Input: 240x320x1 (single-channel depth image)
input_layer = tf.keras.Input(shape=(240, 320, 1))

# Conv Block 1
x = tf.keras.layers.Conv2D(16, (5,5), padding='same', activation='relu', kernel_initializer='he_normal')(input_layer)
x = tf.keras.layers.BatchNormalization()(x)
x = tf.keras.layers.MaxPooling2D((2,2), strides=2)(x)

# Conv Block 2
x = tf.keras.layers.Conv2D(32, (3,3), padding='same', activation='relu', kernel_initializer='he_normal')(x)
x = tf.keras.layers.BatchNormalization()(x)
x = tf.keras.layers.MaxPooling2D((2,2), strides=2)(x)

# Conv Block 3
x = tf.keras.layers.Conv2D(64, (3,3), padding='same', activation='relu', kernel_initializer='he_normal')(x)
x = tf.keras.layers.BatchNormalization()(x)

# Classifier Head
x = tf.keras.layers.Flatten()(x)
x = tf.keras.layers.Dense(512, activation='relu', kernel_initializer='he_normal')(x)
x = tf.keras.layers.Dropout(0.5)(x)
x = tf.keras.layers.Dense(256, activation='relu', kernel_initializer='he_normal')(x)
x = tf.keras.layers.Dropout(0.5)(x)
# Output: 2*num_joints (x,y for each joint), no activation
output_layer = tf.keras.layers.Dense(2*num_joints)(x)

model = tf.keras.Model(input_layer, output_layer)

Give these changes a shot—start with small tweaks (like data normalization and learning rate adjustment) first, then iterate on the network structure. You'll see improvements as you refine the setup!

内容的提问来源于stack exchange，提问作者Harper Long