Aruco标记PnP算法参考点与3D-2D对应关系技术问询

Let's tackle your two questions about ArUco and PnP pose estimation—these are foundational points that trip up a lot of folks getting started with marker-based tracking:

1. What 3D reference points does the PnP algorithm use for ArUco markers? #

The PnP (Perspective-n-Point) algorithm relies exclusively on the four corner points of the ArUco marker as its 3D reference features. These points follow strict conventions in the marker's local coordinate system:

• The marker is assumed to lie flat on the Z=0 plane (all 3D points have a Z-coordinate of 0).
• If your marker has a physical side length of marker_size (e.g., 0.1 meters for a 10cm marker), the four 3D corner coordinates are fixed as:
  • Top-left: (0, 0, 0)
  • Top-right: (marker_size, 0, 0)
  • Bottom-right: (marker_size, marker_size, 0)
  • Bottom-left: (0, marker_size, 0)
    These are known, pre-defined 3D points that you pass to estimatePoseSingleMarkers() (or directly to solvePnP()) along with your camera's calibration data. No other features of the ArUco marker (like its inner binary code) are used for the PnP calculation—only these four corners.

2. How does the algorithm establish 3D-2D correspondences between the real world and the image? #

The 3D-2D matching is a two-step process, mostly handled under the hood by OpenCV's ArUco utilities:

• Step 1: Detect ordered 2D corner points in the image
  estimatePoseSingleMarkers() first calls cv::aruco::detectMarkers() internally to scan the input image. This detector identifies the marker's unique binary pattern, then precisely extracts the four corner points' 2D pixel coordinates. Crucially, the marker's binary code lets the detector determine the exact order of these corners—so it never mixes up, say, the top-left corner with the bottom-right one.
• Step 2: Pair 3D reference points with 2D image points
  The algorithm then matches each pre-defined 3D corner (from the marker's local coordinate system) to its corresponding 2D pixel corner. This correspondence is 1:1 and unambiguous because:
  • The 3D points follow a fixed, consistent order (top-left → top-right → bottom-right → bottom-left).
  • The detectMarkers() function returns the 2D pixel corners in that exact same order.
    Once these correspondences are locked in, solvePnP() uses them to compute the camera's rotation vector (rvec) and translation vector (tvec)—which define the camera's position and orientation relative to the marker's local coordinate system.

内容的提问来源于stack exchange，提问作者Roman Dosaev