如何在OpenCV中正确提取、修改并还原旋转区域内的边界框至原始图像

Hey there, let's work through your issue with re-projecting bounding boxes from rotated/cropped image regions back to the original image. After looking over your code, the core problems are related to incomplete point transformation, coordinate format confusion, and how you're calculating the inverse perspective matrix. Let's break down the fixes step by step.

Key Issues in Your Current Implementation #

• Only Transforming Two Corners: You're just converting the top-left and bottom-right corners of the bounding box. When you reverse a perspective warp, an axis-aligned rectangle becomes a quadrilateral—so you need all four corners to get the correct shape in the original image.
• Mixed Coordinate Formats: Swapping (y,x) and (x,y) incorrectly during transformation leads to shifted or distorted boxes. OpenCV's perspectiveTransform expects points in (x,y) order, so consistency here is critical.
• Inverse Transform Reliability: Using cv2.getPerspectiveTransform(dst_pts, src_pts) can work if point order is perfect, but directly inverting the original perspective matrix with cv2.invert(M) is more reliable and avoids ordering mistakes.

Corrected Code for Bounding Box Re-projection #

Replace your problematic re-projection section with this code, which fixes all the above issues:

# ... (keep all your existing code up to this point)
#=================================================
# Fixed re-projection logic starts here:
#-------------------------------------------- 
nb_box = boxes.shape[0]
color = [(255,0,0), (0,0,255)][no%2]

for i in range(nb_box):
    # Extract the box in (y1, x1, y2, x2) format (matches your TensorFlow output)
    y1, x1, y2, x2 = boxes[i]
    
    # Generate ALL FOUR corners of the axis-aligned box in (x,y) format (required by OpenCV)
    # Order: top-left, top-right, bottom-right, bottom-left
    box_corners = np.array([
        [x1, y1],
        [x2, y1],
        [x2, y2],
        [x1, y2]
    ], dtype=np.float32)
    
    # Calculate the inverse of the original perspective matrix
    # M maps original image -> warped region; inverse maps warped region -> original image
    success, M_inv = cv2.invert(M)
    if not success:
        print(f"Warning: Could not invert transform matrix for contour {no}")
        continue
    
    # Apply inverse perspective transform to all four corners
    transformed_corners = cv2.perspectiveTransform(np.expand_dims(box_corners, axis=0), M_inv)[0]
    # Convert to integer coordinates for drawing
    transformed_corners = np.int0(transformed_corners)
    
    # Draw the transformed quadrilateral on the original image
    cv2.polylines(input_img, [transformed_corners], isClosed=True, color=color, thickness=8)
    
    # Optional: If you need an axis-aligned bounding box around the transformed shape
    # Uncomment the lines below if you don't want the quadrilateral
    # x_min, y_min = np.min(transformed_corners, axis=0)
    # x_max, y_max = np.max(transformed_corners, axis=0)
    # cv2.rectangle(input_img, (x_min, y_min), (x_max, y_max), color, 4)
#end
# ... (keep the rest of your code for saving the final image)

What Changed & Why #

1. Four-Corner Transformation: By converting all four corners of the bounding box, we capture the full warped shape that would appear in the original image—this fixes the distorted or misplaced boxes you were seeing.
2. Reliable Inverse Matrix: Using cv2.invert(M) directly reverses the original warp, ensuring we're applying the exact opposite transformation used to crop the rotated region.
3. Consistent Coordinates: We explicitly define points in (x,y) order, matching OpenCV's requirements, so there's no confusion between row/column (y/x) and x/y coordinate systems.
4. Accurate Drawing: cv2.polylines draws the closed quadrilateral that accurately represents where the bounding box sits in the original image. If you need an axis-aligned box (for downstream processing), uncomment the optional code to compute the min/max of the transformed corners.

内容的提问来源于stack exchange，提问作者Vincent Rougeau-Moss