将Reality Capture变换矩阵转换至目标坐标系的技术求助

我正在使用Reality Capture，已导出场景中20台相机的XMP格式文件，同时拥有目标参考坐标系。尝试将Reality Capture坐标系匹配转换到该参考坐标系，但多次尝试均未成功。

Camera 0的XMP示例 #

<xcr:Rotation>0.9561484049495719 0.2924655797321343 -0.015624096272626363 0.00017677908012416274 -0.053922216042367715 -0.9985451233500854 -0.29288256428395754 0.9547545649479907 -0.05160934265640921</xcr:Rotation>
<xcr:Position>2.75445788094206 -10.557416536967201 15.257369477238798</xcr:Position>

我参考Reality Capture的数学原理，用t = -dot(Rotation, position)构建变换矩阵，但两个坐标系仍存在严重偏差。以下是我的尝试代码：

import numpy as np
from scipy.spatial.transform import Rotation

# RealityCapture rotation matrix and translation vector
realitycapture_rotation = [
    0.9561484049495719, 0.2924655797321343, -0.015624096272626363,
    0.00017677908012416274, -0.053922216042367715, -0.9985451233500854,
    -0.29288256428395754, 0.9547545649479907, -0.05160934265640921
]

realitycapture_translation = [2.75445788094206, -10.557416536967201, 15.257369477238798]

# Convert to a 3x3 matrix
rotation_matrix = np.array(realitycapture_rotation).reshape(3, 3)

# Correct the rotation matrix and translation vector
rotation_matrix_corrected = rotation_matrix[[2, 0, 1], :]
translation_vector_corrected = np.array(realitycapture_translation)[[2, 0, 1]]

# Create a Rotation object from the corrected matrix
rotation = Rotation.from_matrix(rotation_matrix_corrected)

# Calculate the extrinsics matrix
extrinsics = np.concatenate((rotation.as_matrix(), -np.dot(rotation.as_matrix(), translation_vector_corrected[:, np.newaxis])), axis=1)
np.set_printoptions(precision=10, suppress=True)

# Create the full world_from_cam matrix
world_from_cam_realitycapture = np.vstack((extrinsics, np.array([0, 0, 0, 1])))
world_from_cam_realitycapture = np.linalg.inv(world_from_cam_realitycapture)

# Print the RealityCapture 'world_from_cam' matrix
print("Computed 'world_from_cam' matrix from RealityCapture:")
print(world_from_cam_realitycapture)

# Provided 'world_from_cam' matrix
world_from_cam_provided = np.array([
    0.9990035620585193, -0.033809526359390864, 0.02913415387039758, 0.0,
    0.03464496400756332, 0.9989885273263922, -0.02866441590475588, 0.0,
    -0.0281355551447806, 0.02964520530540873, 0.9991644270784941, 0.0,
    0.004656272268761068, -0.0009031649267875787, 0.4233474275953534, 1.0
]).reshape(4, 4)

# Print the provided 'world_from_cam' matrix
print("Provided 'world_from_cam' matrix:")
print(world_from_cam_provided.transpose())

解决方案 #

1. 修正坐标系轴映射 #

Reality Capture默认是Y-Up坐标系，多数目标系统为Z-Up，你之前的轴交换逻辑有误。正确的轴映射规则：

• Reality Capture (X, Y, Z) → 目标系统 (X, -Z, Y)

修改轴修正代码：

# 正确的Y-Up转Z-Up轴映射
rotation_matrix_corrected = rotation_matrix[:, [0, 2, 1]]
rotation_matrix_corrected[:, 1] *= -1  # 反转Z轴符号适配目标坐标系
translation_vector_corrected = np.array([
    realitycapture_translation[0],
    -realitycapture_translation[2],
    realitycapture_translation[1]
])

2. 重构变换矩阵 #

Reality Capture的<xcr:Rotation>是相机到世界的旋转矩阵，<xcr:Position>是相机在世界空间的位置，无需额外求逆操作，直接构建world_from_cam矩阵：

# 直接构建world_from_cam矩阵
world_from_cam = np.eye(4)
world_from_cam[:3, :3] = rotation_matrix_corrected
world_from_cam[:3, 3] = translation_vector_corrected

3. 全局点云配准校准 #

如果单相机校准仍有偏差，用ICP算法计算全局变换：

import open3d as o3d

# 加载Reality Capture稀疏点云和目标坐标系点云
pcd_rc = o3d.io.read_point_cloud("rc_sparse.ply")
pcd_target = o3d.io.read_point_cloud("target_points.ply")

# 执行ICP配准
threshold = 0.02  # 点对匹配阈值
trans_init = np.eye(4)  # 初始变换用轴映射结果
reg_result = o3d.pipelines.registration.registration_icp(
    pcd_rc, pcd_target, threshold, trans_init,
    o3d.pipelines.registration.TransformationEstimationPointToPoint()
)

# 获取全局变换矩阵，应用到所有相机外参
global_transform = reg_result.transformation

4. 验证投影一致性 #

用校准后的相机外参投影RC中的特征点，对比目标坐标系下的对应点坐标，计算误差。若误差在允许范围内，说明变换正确。

内容的提问来源于stack exchange，提问作者Andre Ahmed