如何优化双目相机生成的深度/视差图？已完成校准及高斯滤波

Hey there! Let's break down how you can optimize your depth map quality, building on the preprocessing steps you've already implemented.

First, let's recap what you've done so far: you've calibrated your stereo cameras, applied global histogram equalization, Gaussian blur, and downsampling to your left/right grayscale images. That's a solid foundation—now let's level things up with targeted tweaks and additional steps.

1. Upgrade & Tune Your Stereo Matching Algorithm #

The biggest leap in depth map quality usually comes from choosing the right matching algorithm and tuning its parameters. If you're still using a basic method like cv2.StereoBM, switch to StereoSGBM or StereoBM3D (from OpenCV's ximgproc module)—they handle low-texture regions and edge preservation way better.

Here are key parameters to adjust for your 320x140 resolution:

• numDisparities: Must be a multiple of 16. For your low-res input, start with 16 or 32 (avoid larger values, as they'll introduce more noise).
• blockSize: Use an odd number between 5-9. Smaller blocks preserve fine details, while larger blocks smooth noise in flat areas.
• P1 & P2: These control smoothness. P2 should be 2-4x P1 (try P1=8*3*blockSize² and P2=32*3*blockSize² as a starting point).
• uniquenessRatio: Filters out ambiguous matches—values between 5-15 work well for most cases.

2. Add Post-Processing to Clean Up Noise & Holes #

Even the best matching algorithm will output some noise or empty regions. These post-processing steps will fix that:

• Left-Right Consistency Check: Use OpenCV's disparityWLSFilter to cross-validate disparities from left-to-right and right-to-left. This eliminates mismatched pixels drastically.
• Hole Filling: Use morphological operations like closing (cv2.morphologyEx with cv2.MORPH_CLOSE) to fill small gaps, or cv2.inpaint for larger holes. For edge-preserving smoothing, try a guided filter on the disparity map.
• Valid Depth Clamping: When converting disparity to depth using your calibration parameters (via cv2.reprojectImageTo3D), clamp or remove invalid values (e.g., negative depth values or disparities outside your numDisparities range).

3. Fine-Tune Your Preprocessing Pipeline #

Your current preprocessing is good, but small adjustments can help:

• Replace global histogram equalization with CLAHE (Adaptive Histogram Equalization). Global equalization can over-amplify noise in flat regions—CLAHE adjusts contrast locally, which is better for unevenly lit scenes. Use cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8)).
• Adjust the order of operations: Try downsampling before Gaussian blur and equalization. This reduces computational load and can preserve more texture details in your low-res input.
• Tweak Gaussian blur kernel size: A 5x5 kernel might be enough to reduce noise without blurring away small texture features that stereo matching relies on.

Example Code Snippet (SGBM + WLS Filter) #

Here's a quick example integrating these steps into your workflow:

import cv2
import numpy as np

# Assume you have your calibrated camera parameters (from stereo calibration)
cameraMatrix1, distCoeffs1 = ...  # Left camera params
cameraMatrix2, distCoeffs2 = ...  # Right camera params
R, T = ...  # Rotation/translation between cameras

# Compute Q matrix for disparity-to-depth conversion
Q = cv2.stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, (320,140), R, T)[4]

# Initialize StereoSGBM
sgbm = cv2.StereoSGBM_create(
    minDisparity=0,
    numDisparities=16,
    blockSize=5,
    P1=8*3*5**2,
    P2=32*3*5**2,
    disp12MaxDiff=1,
    uniquenessRatio=10,
    speckleWindowSize=100,
    speckleRange=32
)

# Compute disparities
disparity_left = sgbm.compute(grayLeft, grayRight)
disparity_left = disparity_left.astype(np.float32) / 16.0  # Convert to float disparity

# WLS Filter for post-processing
right_matcher = cv2.ximgproc.createRightMatcher(sgbm)
disparity_right = right_matcher.compute(grayRight, grayLeft)
wls_filter = cv2.ximgproc.createDisparityWLSFilter(sgbm)
disparity_filtered = wls_filter.filter(disparity_left, grayLeft, disparity_map_right=disparity_right)

# Convert to depth map
depth_map = cv2.reprojectImageTo3D(disparity_filtered, Q)[:, :, 2]
depth_map[depth_map < 0] = 0  # Remove invalid depth values

Initial 320x140 resolution depth map:

Start by testing the StereoSGBM + WLS filter combo—it should give you a noticeable improvement over your current output. Then iterate on parameters based on your specific scene (e.g., if you're capturing indoor vs outdoor environments).

内容的提问来源于stack exchange，提问作者gaetano ma