基于Caffe的人脸分类神经网络低精度问题排查求助

Alright, let's figure out why your face classification model is underperforming while it works perfectly on MNIST. The core issue is that face data is far more complex than handwritten digits, and your current setup isn't matched to that complexity. Here are the key problems and actionable fixes:

1. Your network structure is too simple to capture face features #

MNIST digits are low-resolution, high-contrast, and have very distinct, uniform shapes—two fully connected layers are enough to learn these simple patterns. But face images involve subtle variations in lighting, pose, expression, and texture, which require a network that can extract local spatial features (like edges, facial contours, and texture patches).

Fully connected layers only learn global pixel correlations, which can't capture these critical local patterns. You need to switch to a Convolutional Neural Network (CNN). Even a basic CNN adapted from LeNet will drastically improve performance. Here's a quick example of how to modify your network:

# Replace the initial fully connected layers with conv + pooling blocks
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  param { lr_mult: 1 }
  param { lr_mult: 2 }
  convolution_param {
    num_output: 20
    kernel_size: 5
    stride: 1
    weight_filler { type: "xavier" }
    bias_filler { type: "constant" }
  }
}
layer {
  name: "pool1"
  type: "Pooling"
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "conv2"
  type: "Convolution"
  bottom: "pool1"
  top: "conv2"
  param { lr_mult: 1 }
  param { lr_mult: 2 }
  convolution_param {
    num_output: 50
    kernel_size: 5
    stride: 1
    weight_filler { type: "xavier" }
    bias_filler { type: "constant" }
  }
}
layer {
  name: "pool2"
  type: "Pooling"
  bottom: "conv2"
  top: "pool2"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
# Now connect to your fully connected layers (adjust num_output as needed)
layer {
  name: "ip1"
  type: "InnerProduct"
  bottom: "pool2"
  top: "ip1"
  param { lr_mult: 1 }
  param { lr_mult: 2 }
  inner_product_param {
    num_output: 256
    weight_filler { type: "xavier" }
    bias_filler { type: "constant" }
  }
}

2. Data preprocessing is insufficient for face data #

You only resized and scaled your images, but face data needs more targeted preprocessing to help the model learn consistent features:

• Face alignment: If your dataset includes unaligned faces (different head poses, eye positions), the model can't learn a unified representation. You should first detect facial landmarks and align all faces to a standard template (e.g., align eyes and nose to fixed positions).
• Mean normalization: Add mean subtraction to your transform_param to center pixel values around zero. You can compute the RGB mean of your training set and use mean_file or mean_value in the Data layer:

  transform_param {
    scale: 0.00390625
    mean_value: 123.68  # Example R mean
    mean_value: 116.779 # Example G mean
    mean_value: 103.939 # Example B mean
  }
  

• Data augmentation: Add random flips, crops, and brightness adjustments to increase data diversity and prevent overfitting:

  transform_param {
    scale: 0.00390625
    mirror: true # Random horizontal flip
    crop_size: 80 # Random crop to 80x80 (from 100x100 resized images)
  }
  

3. Potential dataset issues to verify #

Double-check your dataset setup—small errors here can completely break training:

• Label format: Ensure your labels_train.txt and labels_test.txt use 0-based indexing for classes (since you have 155 classes, valid indices are 0 to 154). If you used 1-based indexing, the model will try to predict a class that doesn't exist, leading to random guesses.
• LMDB validity: Confirm your LMDB datasets were generated correctly. You can use Caffe's tools or a simple Python script to inspect a few samples and verify that images match their labels.
• Class balance: Even if you aimed for ~260 training images per class, check if any classes have far fewer samples. Imbalanced data can make the model favor dominant classes and perform poorly on rare ones.

4. Hyperparameters need tuning for face data #

The hyperparameters that work for MNIST aren't ideal for face data:

• Learning rate: A base LR of 0.01 is too high for complex face features. Try reducing it to 0.001, and use a step learning rate policy instead of inv to decay the LR as training progresses:

  lr_policy: "step"
  gamma: 0.1
  stepsize: 10000
  

• Iteration count: 30k iterations might be enough for MNIST, but face classification requires more training to learn subtle features. Increase max_iter to 50k or 100k once you switch to a CNN.

Start with switching to a CNN structure—even a simple one will make a massive difference. Then fix your preprocessing, verify your dataset, and adjust hyperparameters. You should see a clear improvement in loss and accuracy once these changes are applied.

内容的提问来源于stack exchange，提问作者user9218823