基于视频的二值图像分类模型优化问题咨询

Great job getting your binary classifier working well on frames with people—let's sort out that edge case where it's spitting out confident 0/1 predictions even when there's no one in the shot. Here are practical, actionable solutions tailored to your TensorFlow/Keras setup:

1. Add a Third "No Person" Class (Most Robust Long-Term Fix) #

The root issue is your model only knows two states, so it's forced to pick one even when neither applies. Expand your problem to a 3-class classification task:

• Collect training data: Gather frames from your video stream that have no people (empty backgrounds, irrelevant objects, etc.) and label them as class 2 (distinct from 0=knee straight, 1=knee bent).
• Modify your model: Swap your final dense layer from a single-unit sigmoid output to a 3-unit softmax output:

  # Replace this:
  model.add(Dense(1, activation='sigmoid'))
  # With this:
  model.add(Dense(3, activation='softmax'))
  

• Update training config: Use SparseCategoricalCrossentropy() as your loss function (if labels are integer-encoded) or CategoricalCrossentropy() (if using one-hot labels). During inference, take the class with the highest probability—if it's the "No Person" class, skip outputting a knee bend/straight prediction.

2. Implement a Confidence Threshold Quick Fix #

If you want to avoid retraining entirely, add a post-processing step to filter out low-confidence predictions:

• After getting the binary output probability p (e.g., probability of knee bend), only act on the prediction if p > 0.9 or p < 0.1 (adjust these thresholds based on your validation data).
• For probabilities in the middle range (e.g., 0.1–0.9), output a "No valid person detected" signal instead of forcing a bend/straight call.
• Note: This is a band-aid—your model might still overconfidently predict 0 or 1 for some empty frames, so pair this with data augmentation of empty frames during retraining if possible.

3. Prepend a Lightweight Person Detector #

Add a preliminary check to ensure there's a person in the frame before running your knee classifier:

• Use a tiny, fast object detector like YOLOv8n, MobileNetSSD, or a TensorFlow Hub pre-trained person detector. These are optimized for speed, which is critical for continuous video streams.
• In your inference pipeline:
  1. Run the person detector on the incoming frame.
  2. If no person is detected, skip the knee classifier and output "No person".
  3. If a person is detected, crop the bounding box around them and feed that cropped image to your knee classifier for the bend/straight prediction.
• This decouples the "person presence" check from the knee state classification, making your system more modular and reliable.

4. Adjust Training Data & Loss for Ambiguous Cases #

If you can't add a third class, retrain your model to recognize ambiguity:

• Add empty frames to training: Include frames with no people, but label them with a "neutral" value (e.g., 0.5 instead of 0 or 1) and use label smoothing in your loss function. This teaches the model to output low-confidence probabilities when the input doesn't match either of the original classes.
• Use focal loss: Modify your binary crossentropy loss to downweight easy examples (like clear bend/straight frames) and focus on hard cases (like empty frames or ambiguous shots). This helps the model learn to be less overconfident on out-of-distribution data.

内容的提问来源于stack exchange，提问作者jKraut