图像旋转后目标关键点映射错误问题求助

问题背景 #

原流程中，按多边形裁剪图像并调整关键点坐标的逻辑正常，但对整张图像进行任意角度旋转后，使用变换矩阵重新映射多边形与关键点坐标时，出现关键点偏移问题，所有旋转角度都会触发该问题。

原正常裁剪关键点代码 #

x,y,w,h = cv2.boundingRect(points_poly_int)
cropped_img = img[y:y+h,x:x+w]

head_coords_after_crop = np.asarray([head_coords_old[0] - x, head_coords_old[1] - y])
center_coords_after_crop = np.asarray([center_coords_old[0] - x, center_coords_old[1] - y])

图像旋转实现函数 #

def rotate_image(mat, angle):
    """
    Rotates an image (angle in degrees) and expands image to avoid cropping
    """

    height, width = mat.shape[:2] # image shape has 3 dimensions
    image_center = (width/2, height/2) # getRotationMatrix2D needs coordinates in reverse order (width, height) compared to shape

    rotation_mat = cv2.getRotationMatrix2D(image_center, angle, 1.)

    # rotation calculates the cos and sin, taking absolutes of those.
    abs_cos = abs(rotation_mat[0,0]) 
    abs_sin = abs(rotation_mat[0,1])

    # find the new width and height bounds
    bound_w = int(height * abs_sin + width * abs_cos)
    bound_h = int(height * abs_cos + width * abs_sin)

    # subtract old image center (bringing image back to origo) and adding the new image center coordinates
    rotation_mat[0, 2] += bound_w/2 - image_center[0]
    rotation_mat[1, 2] += bound_h/2 - image_center[1]

    # rotate image with the new bounds and translated rotation matrix
    rotated_mat = cv2.warpAffine(mat, rotation_mat, (bound_w, bound_h))
    return rotated_mat, rotation_mat 

出错的旋转后坐标映射代码 #

img_roated, C = rotate_image(img, 180)

#Remap polygons coordinates
ones = np.ones((points_poly.shape[0], 1))
new_poly = np.hstack((points_poly,ones))
new_poly = (C @ new_poly.T).T
new_poly =  new_poly.astype(np.int32)

#Crop by new polygons
x,y,w,h = cv2.boundingRect(new_poly)
cropped_img = img_roated[y:y+h,x:x+w]

#Reamp keypoints coordinates
head_coords_new = np.asarray([756.600, 1687.900, 1])
center_coords_new = np.asarray([762.300, 1708.400, 1])
head_coords_new = (C @ head_coords_new.T).T
center_coords_new = (C @ center_coords_new.T).T

head_coords_new = np.asarray([head_coords_old[0] - x, head_coords_old[1] - y])
center_coords_new = np.asarray([center_coords_old[0] - x, center_coords_old[1] - y])

head_coords_new =  head_coords_new.astype(np.int32)
center_coords_new =  center_coords_new.astype(np.int32)

问题分析与修复方案 #

核心错误点 #

1. 变量名误用：完成关键点的旋转变换后，错误使用head_coords_old/center_coords_old计算裁剪偏移，应当使用旋转后的head_coords_new/center_coords_new。
2. 精度损失：过早将坐标转为整数，应先完成裁剪偏移计算，再做类型转换以减少精度丢失。

修复后的旋转后坐标映射代码 #

img_rotated, C = rotate_image(img, 180)

# 重新映射多边形坐标
ones = np.ones((points_poly.shape[0], 1))
new_poly = np.hstack((points_poly, ones))
new_poly = (C @ new_poly.T).T
new_poly = new_poly.astype(np.int32)

# 根据新多边形裁剪图像
x, y, w, h = cv2.boundingRect(new_poly)
cropped_img = img_rotated[y:y+h, x:x+w]

# 重新映射关键点坐标
head_coords_new = np.asarray([756.600, 1687.900, 1])
center_coords_new = np.asarray([762.300, 1708.400, 1])
# 应用旋转变换
head_coords_new = (C @ head_coords_new.T).T
center_coords_new = (C @ center_coords_new.T).T

# 使用旋转后的坐标计算裁剪偏移（修正变量名错误）
head_coords_new = np.asarray([head_coords_new[0] - x, head_coords_new[1] - y])
center_coords_new = np.asarray([center_coords_new[0] - x, center_coords_new[1] - y])

# 最后转为整数类型（调整转换时机）
head_coords_new = head_coords_new.astype(np.int32)
center_coords_new = center_coords_new.astype(np.int32)

额外验证建议 #

• 可先跳过裁剪步骤，直接在旋转后的图像上绘制旋转后的关键点与多边形，验证映射是否正确后再执行裁剪。
• 对于非90度整数倍的旋转，建议保留浮点坐标到最后一步再转整数，避免累积精度误差。

内容的提问来源于stack exchange，提问作者Vivern202