StereoBM在静态图像与视频输出中结果差异的技术求助

Hey there, let's break down why your static disparity maps work great but real-time camera frames are giving you chaos. I've worked with Raspberry Pi stereo setups before, so here are the most likely culprits and fixes to try:

1. Missing Real-Time Frame Rectification #

Your static test images are already rectified (calibrated and aligned), but I suspect your real-time camera frames aren't getting the same treatment. Even with a pre-calibrated hardware kit, you need to apply the calibration parameters to every live frame:

• Double-check if you're running the same distortion correction and stereo rectification on live frames that you used for your static test images.
• If you haven't added this step, grab your pre-calibrated leftMapX, leftMapY, rightMapX, rightMapY matrices and apply them to each split frame:

  # Add this right after splitting left/right frames
  imgLeft = cv2.remap(imgLeft, leftMapX, leftMapY, cv2.INTER_LINEAR)
  imgRight = cv2.remap(imgRight, rightMapX, rightMapY, cv2.INTER_LINEAR)
  

StereoBM relies on perfectly horizontally aligned left/right frames—skip this step, and your matching will be completely off.

2. Mismatched Camera Exposure/White Balance #

Static images are captured in a controlled setting, but live camera frames often have auto-exposure/white balance enabled, which causes left/right frames to have inconsistent brightness/color. This destroys StereoBM's block-matching accuracy:

• Force fixed exposure and white balance settings for your camera:

  # Disable auto-exposure and set fixed values
  cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 0.25)
  cap.set(cv2.CAP_PROP_EXPOSURE, 100) # Adjust this value based on your lighting
  # Disable auto-white balance
  cap.set(cv2.CAP_PROP_AUTO_WB, 0)
  cap.set(cv2.CAP_PROP_WB_TEMPERATURE, 4500) # Neutral white balance
  

• Also, convert your frames to grayscale first (StereoBM works best with single-channel images) and add a light Gaussian blur to reduce camera noise:

  imgLeft_gray = cv2.cvtColor(imgLeft, cv2.COLOR_BGR2GRAY)
  imgRight_gray = cv2.cvtColor(imgRight, cv2.COLOR_BGR2GRAY)
  imgLeft_gray = cv2.GaussianBlur(imgLeft_gray, (3,3), 0)
  imgRight_gray = cv2.GaussianBlur(imgRight_gray, (3,3), 0)
  

3. Frame Splitting Alignment Issues #

Your IOutils.get_split_cam function might be splitting the combined frame incorrectly, leading to misaligned left/right images:

• Test this by displaying imgLeft and imgRight side-by-side. Look for a distant object—its horizontal position should be consistent across both frames. If it's shifted, your split logic has a bug (check the width calculation for splitting the combined frame).

4. Parameter Tuning for Live Frames #

Static image parameters don't always translate perfectly to live frames. Try tweaking these to improve stability:

• Block Size: Increase from 5 to 9 or 11 to make matching more robust to noise (tradeoff: slightly lower resolution).
• Texture Threshold: Lower this value if your live scene has low texture—this helps StereoBM find matches in less detailed areas.
• Speckle Filters: Increase speckleWindowSize and speckleRange to filter out noise-induced false disparities in live frames.

5. WLS Filter Red Herring? #

Before blaming the WLS filter, test the raw dispL output first. Comment out the WLS filtering code and display the raw disparity map. If it's still chaotic, the problem is in the StereoBM setup, not the post-processing.

Quick Code Adjustment Example #

Here's how to integrate the key fixes into your existing code:

sbm = cv2.StereoBM_create(numDisparities=16, blockSize=5)
# ... your existing parameter settings ...

cap = cv2.VideoCapture(0)
res_w = w
res_l = l
cap.set(3,res_w*2)
cap.set(4,res_l)
# Add fixed camera settings here
cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 0.25)
cap.set(cv2.CAP_PROP_EXPOSURE, 100)
cap.set(cv2.CAP_PROP_AUTO_WB, 0)
cap.set(cv2.CAP_PROP_WB_TEMPERATURE, 4500)

while True:
    # Get and split frames
    imgLeft, imgRight = IOutils.get_split_cam(cap,gray=False)
    # Apply rectification
    imgLeft = cv2.remap(imgLeft, leftMapX, leftMapY, cv2.INTER_LINEAR)
    imgRight = cv2.remap(imgRight, rightMapX, rightMapY, cv2.INTER_LINEAR)
    # Preprocess
    imgLeft_gray = cv2.cvtColor(imgLeft, cv2.COLOR_BGR2GRAY)
    imgRight_gray = cv2.cvtColor(imgRight, cv2.COLOR_BGR2GRAY)
    imgLeft_gray = cv2.GaussianBlur(imgLeft_gray, (3,3), 0)
    imgRight_gray = cv2.GaussianBlur(imgRight_gray, (3,3), 0)
    
    # Compute disparities
    dispL = sbm.compute(imgLeft_gray, imgRight_gray)
    # ... rest of your code ...

Start with rectification and fixed camera settings—those are the most common issues with Pi stereo setups. Let me know if any of these steps help!

内容的提问来源于stack exchange，提问作者Theonolev