使用OpenCV相机校准后去畸变图像仍失真，求技术排查

问题背景 #

我是计算机视觉新手，正跟随在线教程学习图像去畸变与相机校准，编写了如下Python代码：

# python imports
import cv2
import numpy as np
import logging

logging.basicConfig(level=logging.INFO, format='%(asctime)s:%(message)s')

# Arrays to store the locations of points where two black squares intersect for the object and for the image.
objectpoints = []
imagepoints = []

# Prepare the points where the black squares intersect like (0,0,0), (1,0,0), (2,0,0)...... The third column denotes the z-coordinate of the point. So, it is always zero.

# 1. The first step is to create an array that will be correct shape to store the object coordinates. 
objp = np.zeros(shape=(6*8, 3), dtype=np.float32)

# 2. We will need to populate this array with correct coordinates. If you do not understand the math behind it, take a pen and paper and write it. It is easy. Just pay attention to the 
# type of the matrix formed by mgrid[] and the shape of the matrix. And print the final matrix to see how the result looks like at every operation on RHS. 
# These points indicate the points where the two black rectangeles touch on an actual (undistorted) chessboard. 
 
objp[:,:2]= np.mgrid[0:8, 0:6].T.reshape(-1, 2)

logging.info("Created object points for chessboard")

# Read any one image form the distorted images and convert it to gray-scale to feed it to the findChessboardCorners() in cv2.
img = cv2.imread(r"D:\Udacity course\Module2_camera\corner detection and camera calibration\calibration_test.png")
img_gray = cv2.cvtColor(img, cv2.IMREAD_GRAYSCALE)


found_corners, corners = cv2.findChessboardCorners(img_gray, (8,6), None)
logging.info("Found corners for input image")

# Append all the object points to objecpoints array and all image points to imagepoints array. (If the cv2.findChessboardCorners() is able to detect the corners in the image)
if found_corners == True:
    imagepoints.append(corners)
    objectpoints.append(objp)
    logging.info("loaded object points and image points to respective lists")

else:
    logging.error("No corners found in the image. Check the input image and the parameters for the image such as horizontal and vertical columns")

# In the following steps we will find the camera calibration matrix includig the distortion coefficient (dist), camera matrix (mtx), the radial and tangential skew vectors.
# It will also return the position of the camera in the world with values for rotation vector (rvec) and the translation vectors (tvecs)
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objectpoints, imagepoints, img.shape[::2], None, None)

logging.info("Found the camera matrix and the distortion coefficient from the image.")
logging.info("Camera matrix \n{}".format(mtx))
logging.info("distortion coefficient\n{}".format(dist))

# We can take in distortion image and return the undistorted image with the help of distortion coefficient and the camera matrix.
dst = cv2.undistort(img, mtx, dist, None, mtx)
cv2.imwrite('undistorted_img.png', dst)
cv2.imshow('undistorted image', dst)
cv2.waitKey(0)

运行后得到的去畸变图像底部仍存在畸变，输出的相机矩阵与畸变系数如下：

2023-03-05 00:03:50,596:Created object points for chessboard
2023-03-05 00:03:50,642:Found corners for input image
2023-03-05 00:03:50,643:loaded object points and image points to respective lists
2023-03-05 00:03:50,650:Found the camera matrix and the distortion coefficient from the image.
2023-03-05 00:03:50,651:Camera matrix
[[1.05424776e+03 0.00000000e+00 5.76945956e+02]
 [0.00000000e+00 9.85451700e+02 7.42761812e+01]
 [0.00000000e+00 0.00000000e+00 1.00000000e+00]]
2023-03-05 00:03:50,652:distortion coefficient
[[-0.390704   -1.37769503  0.19748877  0.03508645  1.58862494]]

原始畸变图像：

去畸变图像：

请问问题出在哪里？是否存在我未理解的核心概念？

问题原因与核心概念解析 #

1. 单张图像校准的局限性 #

相机校准的核心是通过多视角棋盘格图像拟合相机内参（畸变系数、相机矩阵），单张图像提供的样本点和视角信息严重不足：

• 仅用1张图无法覆盖镜头的全畸变范围，尤其是图像边缘/底部这类畸变更明显的区域
• 单张图的校准参数拟合误差大，无法准确描述镜头的全局畸变规律

2. 棋盘格特征点的覆盖缺失 #

从原始图像看，棋盘格仅占据图像上半部分，底部完全没有棋盘格角点。cv2.calibrateCamera只能基于已检测到的特征点计算参数，没有特征点的区域，算法无法推断畸变规律，导致去畸变后底部残留畸变。

3. 校正优化方案 #

• 采集多视角棋盘格图像：至少拍摄10-20张，让棋盘格出现在图像的不同位置（左上、右下、底部、边缘等），确保覆盖整个画面
• 细化角点检测精度：在findChessboardCorners后添加cv2.cornerSubPix提升角点坐标准确性：

  if found_corners == True:
      criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
      corners = cv2.cornerSubPix(img_gray, corners, (11,11), (-1,-1), criteria)
      imagepoints.append(corners)
      objectpoints.append(objp)
  

• 使用最优相机矩阵优化校正：用cv2.getOptimalNewCameraMatrix替换原相机矩阵，可裁剪无效黑边并提升校正精度：

  h, w = img.shape[:2]
  newcameramtx, roi = cv2.getOptimalNewCameraMatrix(mtx, dist, (w,h), 1, (w,h))
  dst = cv2.undistort(img, mtx, dist, None, newcameramtx)
  # 裁剪有效区域
  x, y, w_roi, h_roi = roi
  dst = dst[y:y+h_roi, x:x+w_roi]
  

内容的提问来源于stack exchange，提问作者programmer_04_03