基于Python OpenCV与ZED 2实现视差图（禁用ZED SDK）的优化求助

问题背景 #

课程作业要求禁用ZED SDK，仅用OpenCV实现ZED2相机的视差图生成。处理官方示例图像正常，但自行采集的ZED2图像生成的视差图效果差，已尝试调整参数无明显改善，现有代码如下：

import cv2
import numpy as np
from matplotlib import pyplot as plt

# Load the image in grayscale
image = cv2.imread('TestImage4.png', 0)

# Split the image into left and right halves
left_right_image = np.split(image, 2, axis=1)

# Display the left and right images
cv2.imshow("Left Image", left_right_image[0])
cv2.imshow("Right Image", left_right_image[1])
cv2.waitKey(0)
cv2.destroyAllWindows()

# Parameters for the StereoSGBM algorithm
block_size = 11
min_disp = 0
max_disp = 255
num_disp = max_disp - min_disp
uniquenessRatio = 5
speckleWindowSize = 200
speckleRange = 2
disp12MaxDiff = 0

# Create the StereoSGBM object
stereo = cv2.StereoSGBM_create(
    minDisparity=min_disp,
    numDisparities=num_disp,
    blockSize=block_size,
    uniquenessRatio=uniquenessRatio,
    speckleWindowSize=speckleWindowSize,
    speckleRange=speckleRange,
    disp12MaxDiff=disp12MaxDiff,
    P1=8 * 3 * block_size ** 2,
    P2=32 * 3 * block_size ** 2,
    mode=cv2.STEREO_SGBM_MODE_SGBM
)

# Compute the disparity map
disparity = stereo.compute(left_right_image[0], left_right_image[1])

# Normalize the disparity map for displaying
disparity = cv2.normalize(disparity, disparity, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX)
disparity = np.uint8(disparity)

# Resize the disparity map for better viewing
disparity_resized = cv2.resize(disparity, (disparity.shape[1] // 2, disparity.shape[0] // 2))

# Display the disparity map
cv2.imshow("Disparity", disparity_resized)
cv2.waitKey(0)
cv2.destroyAllWindows()

优化建议 #

1. 先对左右图像做畸变校正

ZED2相机采集的原始图像存在径向畸变和切向畸变，未校正的图像极线不平行，会导致立体匹配大量错误。需通过相机内参和畸变系数校正图像：

• 从ZED官网或相机标定获取ZED2的内参（fx, fy, cx, cy）和畸变系数（k1, k2, p1, p2, k3）
• 用cv2.undistort分别校正左右图像：

# 示例内参（需替换为实际ZED2标定参数）
left_camera_matrix = np.array([[fx, 0, cx], [0, fy, cy], [0, 0, 1]], dtype=np.float32)
left_dist_coeffs = np.array([k1, k2, p1, p2, k3], dtype=np.float32)
right_camera_matrix = left_camera_matrix.copy()
right_dist_coeffs = left_dist_coeffs.copy()

# 校正左右图像
left_img = cv2.undistort(left_right_image[0], left_camera_matrix, left_dist_coeffs)
right_img = cv2.undistort(left_right_image[1], right_camera_matrix, right_dist_coeffs)

2. 修正SGBM参数的错误配置

• numDisparities必须是16的整数倍：当前num_disp=255不符合要求，会导致算法运行异常，建议改为max_disp=256，num_disp=256（或根据场景选128/192）
• 调整核心匹配参数：
  • block_size：纹理丰富场景用5-9，纹理稀疏场景用11-15（必须为奇数）
  • uniquenessRatio：提高到10-15，过滤低置信度匹配点
  • min_disp：根据ZED2基线（120mm）和焦距，可设为16，减少无效近距离匹配
  • P2：增大到64 * 3 * block_size ** 2，增强视差图平滑性（P2需远大于P1）
  • mode：改用cv2.STEREO_SGBM_MODE_HH，半全局匹配的高精度模式

修正后的参数示例：

block_size = 9
min_disp = 16
max_disp = 256
num_disp = max_disp - min_disp  # 240，是16的倍数
uniquenessRatio = 15
speckleWindowSize = 100
speckleRange = 1
disp12MaxDiff = 1

stereo = cv2.StereoSGBM_create(
    minDisparity=min_disp,
    numDisparities=num_disp,
    blockSize=block_size,
    uniquenessRatio=uniquenessRatio,
    speckleWindowSize=speckleWindowSize,
    speckleRange=speckleRange,
    disp12MaxDiff=disp12MaxDiff,
    P1=8 * 3 * block_size ** 2,
    P2=64 * 3 * block_size ** 2,
    mode=cv2.STEREO_SGBM_MODE_HH
)

3. 增加视差图后处理

用WLS滤波优化视差图，利用左图的边缘信息减少噪声和模糊：

# 计算视差后转换为float32（WLS需要）
disparity = stereo.compute(left_img, right_img).astype(np.float32) / 16.0

# 创建WLS滤波器
wls_filter = cv2.ximgproc.createDisparityWLSFilter(stereo)
right_matcher = cv2.ximgproc.createRightMatcher(stereo)
disparity_right = right_matcher.compute(right_img, left_img).astype(np.float32) / 16.0

# 应用滤波
filtered_disp = wls_filter.filter(disparity, left_img, disparity_map_right=disparity_right)

# 归一化显示
filtered_disp = cv2.normalize(filtered_disp, filtered_disp, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX)
filtered_disp = np.uint8(filtered_disp)

4. 确保图像分割准确性

避免依赖np.split，手动指定左右图的边界，防止图像宽度为奇数时的分割偏移：

img_width = image.shape[1]
left_img = image[:, :img_width//2]
right_img = image[:, img_width//2:]

内容的提问来源于stack exchange，提问作者gyan fransen