OpenCV perspectiveTransform函数调用失败问题求助

问题背景 #

尝试使用OpenCV的perspectiveTransform函数对3D边界框点执行外参变换，但始终报错，调整数据类型和矩阵尺寸后仍未解决。

初始代码实现 #

主逻辑代码 #

// 获取外参变换信息
cv::Mat extrinsicCalibration = getTranformations(std::string("..\ExtrinsicCalibration\" + sensor_3.serialNo + "\transformation_Mat.yml")).inv();

// 将边界框点转换为cv::Mat
cv::Mat points_to_transform = convertToHomogeneousCoords(minBound, maxBound);

// 对外参变换后的3D边界框点应用变换以从相机视角呈现
cv::Mat transformed_points(4, 2, CV_32F);
std::cout << points_to_transform.type() << "  --  "
          << points_to_transform.size() << "\n"
          << extrinsicCalibration.type() << "  --  "
          << extrinsicCalibration.size() << "\n"
          << transformed_points.type() << "  --  "
          << transformed_points.size() << "\n";
try {
    cv::perspectiveTransform(points_to_transform, transformed_points, extrinsicCalibration);
} catch (cv::Exception& e) {
    std::cout << "Exception caught: " << e.what() << std::endl;
}

齐次坐标转换函数 #

cv::Mat convertToHomogeneousCoords(const glm::vec3& point1, const glm::vec3& point2) {
    cv::Mat points_to_transform(4, 2, CV_32F);
    points_to_transform.at<float>(0, 0) = point1.x;
    points_to_transform.at<float>(1, 0) = point1.y;
    points_to_transform.at<float>(2, 0) = point1.z;
    points_to_transform.at<float>(3, 0) = 1.0f; // 设置w分量为1
    points_to_transform.at<float>(0, 1) = point2.x;
    points_to_transform.at<float>(1, 1) = point2.y;
    points_to_transform.at<float>(2, 1) = point2.z;
    points_to_transform.at<float>(3, 1) = 1.0f; // 设置w分量为1
    return points_to_transform;
}

初始运行信息 #

打印的矩阵类型与尺寸 #

5  --  [2 x 4]  
5  --  [4 x 4]  
5  --  [2 x 4] 

抛出的异常 #

Unhandled exception at 0x00007FFF680640AC in RAU_Control_Interface_Software.exe: Microsoft C++ exception: cv::Exception at memory location 0x00000032570FD7F0. 

Exception caught: OpenCV(4.6.0) C:\Users\Mikke\source\repos\opencv-4.6.0\modules\core\src\matmul.dispatch.cpp:550: error: (-215:Assertion failed) scn + 1 == m.cols in function 'cv::perspectiveTransform'

编辑补充：调整后的代码与问题 #

得知perspectiveTransform会自动处理齐次坐标，修改代码如下：

修改后主逻辑代码 #

// 获取外参变换信息
cv::Mat extrinsicCalibration = getTranformations(std::string("..\ExtrinsicCalibration\" + sensor_3.serialNo + "\transformation_Mat.yml")).inv();

// 将边界框点转换为cv::Mat
cv::Mat points_to_transform(2, 3, CV_32F);
points_to_transform.at<float>(0, 0) = minBound.x;
points_to_transform.at<float>(0, 1) = minBound.y;
points_to_transform.at<float>(0, 2) = minBound.z;
points_to_transform.at<float>(1, 0) = maxBound.x;
points_to_transform.at<float>(1, 1) = maxBound.y;
points_to_transform.at<float>(1, 2) = maxBound.z;
// 对外参变换后的3D边界框点应用变换以从相机视角呈现
cv::Mat transformed_points(2, 3, CV_32F);
std::cout << points_to_transform.type() << "  --  "
          << points_to_transform.size() << "\n"
          << extrinsicCalibration.type() << "  --  "
          << extrinsicCalibration.size() << "\n"
          << transformed_points.type() << "  --  "
          << transformed_points.size() << "\n";
try {
    cv::perspectiveTransform(points_to_transform, transformed_points, extrinsicCalibration);
} catch (cv::Exception& e) {
    std::cout << "Exception caught: " << e.what() << std::endl;
}

修改后打印的矩阵类型与尺寸 #

5  --  [3 x 2]
5  --  [4 x 4]
5  --  [3 x 2]

修改后抛出的异常 #

OpenCV(4.6.0) Error: Assertion failed (scn + 1 == m.cols) in cv::perspectiveTransform, file C:\Users\Mikke\source\repos\opencv-4.6.0\modules\core\src\matmul.dispatch.cpp, line 550
Exception caught: OpenCV(4.6.0) C:\Users\Mikke\source\repos\opencv-4.6.0\modules\core\src\matmul.dispatch.cpp:550: error: (-215:Assertion failed) scn + 1 == m.cols in function 'cv::perspectiveTransform'

OpenCV(4.6.0) Error: Assertion failed ((unsigned)(i1 * DataType<_Tp>::channels) < (unsigned)(size.p[1] * channels())) in cv::Mat::at, file C:\Users\Mikke\source\repos\opencv-4.6.0\modules\core\include\opencv2/core/mat.inl.hpp, line 899

问题分析与解决方法 #

核心问题 #

1. 函数定位错误：perspectiveTransform的设计目标是透视投影（3D→2D）或2D透视变换，并不适用于3D刚体变换（旋转+平移）场景，你使用4x4外参矩阵做3D点变换，本身就不符合该函数的使用场景。
2. 矩阵维度不匹配：该函数要求输入点矩阵为N行1列的N通道矩阵或1行N列的N通道矩阵，你当前的点矩阵是[3x2]（2个点按列存储），不符合输入格式要求；同时断言scn + 1 == m.cols失败，本质是函数不支持用4x4矩阵处理3D点的刚体变换。
3. 访问越界：修改后的代码中Mat::at的越界报错是维度不匹配引发的连锁问题。

正确解决方案 #

如果要执行3D点的刚体变换，应使用cv::transform配合矩阵运算，或手动完成齐次坐标变换，示例代码如下：

方法1：手动齐次坐标矩阵运算

// 获取外参变换信息
cv::Mat extrinsicCalibration = getTranformations(std::string("..\ExtrinsicCalibration\" + sensor_3.serialNo + "\transformation_Mat.yml")).inv();

// 构造2个3D点的矩阵（每个点占一行）
cv::Mat points_to_transform(2, 3, CV_32F);
points_to_transform.at<float>(0, 0) = minBound.x;
points_to_transform.at<float>(0, 1) = minBound.y;
points_to_transform.at<float>(0, 2) = minBound.z;
points_to_transform.at<float>(1, 0) = maxBound.x;
points_to_transform.at<float>(1, 1) = maxBound.y;
points_to_transform.at<float>(1, 2) = maxBound.z;

// 转换为齐次坐标（添加w=1的列）
cv::Mat points_homogeneous;
cv::hconcat(points_to_transform, cv::Mat::ones(2, 1, CV_32F), points_homogeneous);

// 执行矩阵乘法（注意转置外参矩阵，OpenCV为行优先存储）
cv::Mat transformed_points_homogeneous = points_homogeneous * extrinsicCalibration.t();

// 转换回非齐次3D坐标（可选，根据需求保留）
cv::Mat transformed_points = transformed_points_homogeneous.colRange(0,3).clone();

方法2：使用cv::transform配合平移分量

// 获取外参变换信息
cv::Mat extrinsicCalibration = getTranformations(std::string("..\ExtrinsicCalibration\" + sensor_3.serialNo + "\transformation_Mat.yml")).inv();

// 构造2个3D点的矩阵（每个点占一行）
cv::Mat points_to_transform(2, 3, CV_32F);
points_to_transform.at<float>(0, 0) = minBound.x;
points_to_transform.at<float>(0, 1) = minBound.y;
points_to_transform.at<float>(0, 2) = minBound.z;
points_to_transform.at<float>(1, 0) = maxBound.x;
points_to_transform.at<float>(1, 1) = maxBound.y;
points_to_transform.at<float>(1, 2) = maxBound.z;

// 执行旋转变换
cv::Mat transformed_points;
cv::transform(points_to_transform, transformed_points, extrinsicCalibration.rowRange(0,3).colRange(0,3));

// 添加平移分量
cv::Mat translation = extrinsicCalibration.rowRange(0,3).col(3);
transformed_points += cv::repeat(translation.t(), 2, 1);

关键说明 #

• 如果需要将3D点投影到2D图像平面，才是perspectiveTransform的适用场景，此时变换矩阵应为3x4的相机内参+外参组合矩阵，输入为3D点，输出为2D点。
• 3D刚体变换优先使用上述两种方法，避免误用perspectiveTransform。

内容的提问来源于stack exchange，提问作者mikkelsen1996