如何求解相机到OptiTrack动作捕捉坐标系的转换关系？

Hey there! Let’s walk through the complete workflow to figure out the coordinate transformation between your camera (mounted on the tracked rigid body) and the OptiTrack motion capture system. This will let you convert any 3D point from the camera’s frame to the OptiTrack global frame smoothly.

We need to solve for the rigid transformation matrix (consisting of a rotation matrix R and translation vector t) that maps points from the camera coordinate system (Cam) to the OptiTrack coordinate system (OptiTrack). The conversion formula is:

P' = R * P + t

Where P is a 3D point in the camera frame, and P' is its corresponding point in the OptiTrack frame.

Step 1: Clarify Coordinate System Definitions #

First, make sure you have a clear understanding of both frames to avoid confusion:

• OptiTrack Global Frame: Typically a right-handed system where X points forward into the capture volume, Y points left, and Z points upward. You can confirm this in your OptiTrack software (like Motive) under coordinate settings.
• Camera Frame: Standard computer vision right-handed frame: X points right along the sensor, Y points down, and Z points forward out of the lens.

Step 2: Collect Corresponding 3D Points (Critical for Calibration) #

You’ll need a calibration target (either a chessboard or an OptiTrack-marked rigid body) to get pairs of 3D points that exist in both frames:

• Camera Frame Points:
  1. First, calibrate your camera to get its intrinsic parameters (using tools like OpenCV’s calibrateCamera if it’s a monocular camera).
  2. Capture images of the calibration target, then use solvePnP to compute the 3D coordinates of the target’s feature points in the camera frame. If you’re using a depth camera, you can directly retrieve 3D coordinates from the depth map.
• OptiTrack Frame Points:
  1. Place the calibration target in the capture volume. Use OptiTrack’s software to track the target’s rigid body.
  2. Export or record the 3D coordinates of the target’s feature points directly from the OptiTrack system.

  Pro Tip: Use at least 3 non-collinear point pairs to compute the transformation, but for better accuracy, use 10-20 pairs and apply least-squares optimization.

Step 3: Calculate the Rigid Transformation (R & t) #

With your corresponding point pairs, you can compute R and t. The most reliable method uses SVD (Singular Value Decomposition):

Manual SVD Workflow: #

1. Compute the centroid of the camera frame points (C_cam) and OptiTrack frame points (C_opt).
2. Decentralize both point sets by subtracting their centroids:

   p_i_decentralized = p_i - C_cam
   P_i_decentralized = P_i - C_opt
   

3. Construct the covariance matrix H from the decentralized points:

   H = sum(p_i_decentralized * P_i_decentralized.T)
   

4. Perform SVD on H to get U, S, V^T.
5. Compute the rotation matrix R = V * U^T. If the determinant of R is -1 (indicating a mirror flip), flip the sign of the last column of V and recompute R.
6. Calculate the translation vector:

   t = C_opt - R * C_cam
   

Using Pre-built Libraries (Easier & Faster) #

If you’re using Python, leverage libraries like scipy or OpenCV to skip manual calculations:

import numpy as np
from scipy.spatial.transform import Rotation

# Example: cam_points (N,3) = camera frame 3D points; opt_points (N,3) = OptiTrack frame points
cam_centroid = np.mean(cam_points, axis=0)
opt_centroid = np.mean(opt_points, axis=0)

# Decentralize points
cam_decentralized = cam_points - cam_centroid
opt_decentralized = opt_points - opt_centroid

# Compute covariance matrix and SVD
H = cam_decentralized.T @ opt_decentralized
U, S, Vt = np.linalg.svd(H)

# Calculate rotation matrix
R = Vt.T @ U.T
# Ensure proper rotation (no reflection)
if np.linalg.det(R) < 0:
    Vt[-1, :] *= -1
    R = Vt.T @ U.T

# Calculate translation vector
t = opt_centroid - R @ cam_centroid

# Now use P_opt = R @ P_cam.reshape(3,1) + t.reshape(3,1) for point conversion

Step 4: Validate Transformation Accuracy #

Don’t skip this step! Verify your R and t with unused calibration points:

1. Take a point from the camera frame, apply the transformation to get its OptiTrack frame position.
2. Compare this computed position with the actual position measured by OptiTrack.
3. Calculate the RMSE (Root Mean Square Error) between predicted and actual points. If the error is too high, check for mismatched point pairs, target movement during calibration, or poor camera intrinsic calibration, then re-collect data.

Step 5: Real-Time Application #

Once your transformation is validated, you can apply it in real time:

• For any 3D point P in the camera frame, compute its OptiTrack frame equivalent using P' = R @ P + t (ensure correct vector/matrix dimensions).
• Alternative approach: If you’re tracking the rigid body’s pose in OptiTrack (R_rigid, t_rigid) and have pre-calibrated the camera’s pose relative to the rigid body (R_cam_rigid, t_cam_rigid), you can combine the transformations directly:

  R = R_rigid @ R_cam_rigid
  t = R_rigid @ t_cam_rigid + t_rigid
  

  This is useful if you need to reposition the camera on the rigid body later—just re-calibrate the camera-to-rigid-body pose instead of the full camera-to-OptiTrack pose.

内容的提问来源于stack exchange，提问作者ahmad seyfi