使用OpenCV校正振镜扫描系统桶形枕形复合畸变的问题问询

我正在尝试使用OpenCV校正图像畸变，待校正的畸变类型为桶形与枕形叠加的复合畸变，如下所示：

我使用的设备并非普通相机，而是振镜扫描系统，无法按照OpenCV官方教程的方案拍摄棋盘格图案完成标定。
但我可以控制扫描器移动到目标位置，并测量激光束在成像平面的实际位置，测量结果示例如下：

我将上述对应点数据代入OpenCV的calibrateCamera函数，编写校正脚本如下：

import numpy as np
import cv2

targetPosX = np.array([-4., -2., 0., 2., 4., -4., -2., 0., 2., 4., -4., -2., 2., 4., -4., -2., 0., 2., 4., -4., -2., 0., 2., 4.])
targetPosY = np.array([-4., -4., -4., -4., -4., -2., -2., -2., -2., -2., 0., 0., 0., 0., 2., 2., 2., 2., 2., 4., 4., 4., 4., 4.])
actualPosX = np.array([-4.21765834, -2.14708042, -0.07755157, 1.9910175, 4.05941744, -4.17816164, -2.10614537, -0.03775821, 2.02883123, 4.09875409, -4.13937186, -2.07079973, 2.07072068, 4.1377518, -4.10200901, -2.03229052, 0.0367603, 2.10655379, 4.17627114, -4.06449305, -1.99426964, 0.07737988, 2.14365487, 4.21625359])
actualPosY = np.array([-4.04808315, -4.08681247, -4.12545265, -4.16807799, -4.20657896, -1.98568911, -2.0217478, -2.06356789, -2.10326313, -2.14456442, 0.07567631, 0.03889721, -0.04043382, -0.08069954, 2.14054726, 2.09940048, 2.05965315, 2.02167639, 1.9800822, 4.20167787, 4.16215278, 4.12334605, 4.08099448, 4.04376011])

scale = 100 # px / mm
height = 9 * scale # range of measured points is -4 to 4mm --> show area from -4.5 to 4.5 with 100 px / mm
width = 9 * scale

def scale_and_shift(array, scl, shift):
    array *= scl
    array += shift
    return array

# shift recorded positon into image coordinate system
targetPosX = scale_and_shift(targetPosX, scale, width / 2.)
targetPosY = scale_and_shift(targetPosY, scale, height / 2.)
actualPosX = scale_and_shift(actualPosX, scale, width / 2.)
actualPosY = scale_and_shift(actualPosY, scale, height / 2.)

# create images
target_image = np.full((height,width), 255)
combined_image = np.full((height,width), 255)
actual_image = np.full((height,width), 255) 
for i in range(len(targetPosX)):
    cv2.circle(target_image, (int(targetPosX[i]), int(targetPosY[i])), 20, 0, -1)
    
    # circle in combined image is target position, full point is actual position
    cv2.circle(combined_image, (int(targetPosX[i]), int(targetPosY[i])), 20, 0, 5)
    cv2.circle(combined_image, (int(actualPosX[i]), int(actualPosY[i])), 20, 0, -1)

    cv2.circle(actual_image, (int(actualPosX[i]), int(actualPosY[i])), 20, 0, -1)

cv2.imwrite("combined_before.png", combined_image)

# create point lists for calibrateCamera function. set 3rd dimension to zero.
targetPoints = np.array([np.vstack([targetPosX, targetPosY]).T]).astype("float32")
targetPoints_zero = np.array([np.vstack([targetPosX, targetPosY, list(np.zeros(len(targetPosX)))]).T]).astype("float32")
imagePoints = np.array([np.vstack([actualPosX, actualPosY]).T]).astype("float32")
imagePoints_zero = np.array([np.vstack([actualPosX, actualPosY, np.zeros(len(actualPosX))]).T]).astype("float32")

# read image to apply to
# saving and reading because just passing the actual_image somehow didn't work
cv2.imwrite("image.png", actual_image)
img = cv2.imread("image.png", cv2.IMREAD_GRAYSCALE)
h, w = img.shape[:2]

# calulate distortion matrix
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(targetPoints_zero, imagePoints, (h,w), None, None)

# refine distortion matrix to avoid cut-off
newcameramtx, roi = cv2.getOptimalNewCameraMatrix(mtx, dist, (w,h), 1, (w,h))

# undistort
dst = cv2.undistort(img, mtx, dist, None, newcameramtx)

cv2.imwrite('calibresult.png', dst)
cv2.imwrite("correction.png", dst - actual_image)

for i in range(len(targetPosX)):
    # circle in combined image is target position, full point is actual position
    cv2.circle(dst, (int(targetPosX[i]), int(targetPosY[i])), 20, 0, 5)

cv2.imwrite('combined_result.png', dst)

但最终校正结果不符合预期，校正后的图像与目标图像无法对齐（实心点为校正后的实际点，空心圆为目标点）：

对比校正前后的差异可见仅实现了极小幅的畸变补偿（下图为实际图像校正前后的差值图）：

请问是否有方法可以调整calibrateCamera的参数优化校正效果？或是该工具本身不适用于当前场景？

问题原因与解决方法 #

1. 现有方案效果差的核心原因 #

cv2.calibrateCamera是为普通针孔相机设计的标定工具，默认仅求解最多6阶径向畸变+2阶切向畸变参数，适配的是相机镜头产生的常规畸变。振镜扫描系统的桶形+枕形叠加复合畸变不符合常规针孔相机的畸变模型，且你当前输入的对应点数量少，默认参数下求解的低阶畸变系数不足以拟合复杂畸变，自然只能得到极弱的校正效果。

2. 适配振镜场景的最优方案 #

你已经有了实际坐标到目标坐标的一一对应点对，不需要强行套用相机标定工具，直接用更适配的方法即可：

• 仅需要校正点坐标：直接对现有24组点对做二阶/三阶多项式拟合，分别拟合x、y坐标的校正函数，后续采集到的实际点直接代入拟合后的公式计算目标位置即可，精度远高于默认相机标定结果。
• 需要校正整幅扫描图像：使用cv2.remap方法。先根据你所有的对应点对生成全局的x、y坐标映射表（可选用双线性插值或薄板样条插值补充无测量点的区域），再调用cv2.remap对输入图像直接做像素重映射，即可实现高精度的畸变校正。

3. 坚持使用calibrateCamera的调整方案 #

可以尝试开启更高阶的畸变系数求解，调用calibrateCamera时传入高阶畸变求解标志位：

ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(targetPoints_zero, imagePoints, (h,w), None, None, flags=cv2.CALIB_RATIONAL_MODEL)

这个标志位会让函数求解最高8阶的径向畸变系数，更接近复合畸变特征，但效果依然不如直接用点对拟合映射表的方案。

内容的提问来源于stack exchange，提问作者user3696412