三维重建中为何需对图像去畸变并归一化坐标？代码疑问解析

1. Image Undistortion: Fixing Lens Imperfections #

• Cameras don’t capture perfect pinhole images—lenses introduce radial distortion (barrel or pincushion warping) and tangential distortion (shifts from lens misalignment).
• The core math behind 3D reconstruction (like epipolar geometry, triangulation) relies on the ideal pinhole camera model, which doesn’t account for these real-world lens flaws.
• Skipping undistortion would mislocate feature points, break epipolar line alignment, and produce wildly inaccurate 3D point positions—especially noticeable at image edges where distortion is worst.

2. Coordinate Normalization: Boosting Numerical Stability #

• Pixel coordinates are often large (e.g., 1920x1080), which can cause floating-point precision errors when solving linear systems (like those used to compute the Essential Matrix).
• Normalization fixes this with two key steps:
  • Centering: Shift all points so their mean sits at (0,0).
  • Scaling: Rescale points so their average distance from the origin is √2, ensuring the data has unit variance.
• This step makes algorithms more robust to noise and reduces numerical instability, leading to far more accurate matrix calculations.

Why Undistort First in the Code? #

The lines:

self.img1 = cv2.undistort(self.img1, self.K, self.d)
self.img2 = cv2.undistort(self.img2, self.K, self.d)

are non-negotiable because every subsequent step depends on pixel positions aligned with the ideal pinhole model:

• Feature matching: Distorted images would place features in incorrect locations, leading to bad matches.
• Essential Matrix calculation: This matrix relies on relationships between normalized camera coordinates (derived from undistorted pixels) across two views. Distorted pixels would make these normalized coordinates invalid, rendering E meaningless.
• Undistorting first converts raw camera images into the idealized projection our 3D reconstruction math expects.

Essential Matrix: What It Does and How It Fits In #

After undistortion, here’s the typical flow (and what the code is likely executing next):

1. Feature Detection & Matching: Find corresponding points between img1 and img2 (e.g., using SIFT or ORB).
2. Normalize Pixel Coordinates: Convert matched pixel points (u, v) to normalized camera coordinates using the intrinsic matrix self.K:

   # For a single point (u, v)
   x = (u - self.K[0][2]) / self.K[0][0]
   y = (v - self.K[1][2]) / self.K[1][1]
   

   This removes the effect of camera intrinsics, leaving points in the normalized camera plane.
3. Compute Essential Matrix: Use the normalized corresponding points to calculate E (usually via the 8-point algorithm with RANSAC to filter outliers). E encodes the relative rotation (R) and translation (t) between the two camera positions.
4. Decompose E: The Essential Matrix can be split into four possible (R, t) pairs. We validate which pair results in 3D points with positive depth (points lie in front of both cameras) to get the correct camera pose.
5. Triangulation: Use the valid (R, t) and matched points to compute the 3D coordinates of scene points.

In short, undistortion is the foundation—without it, all downstream geometric computations would be based on incorrect pixel positions, leading to a faulty 3D reconstruction.

内容的提问来源于stack exchange，提问作者Luisa