GCP Document AI预处理后边界框与原图对齐问题求助

GCP Document AI会对图像进行预处理以消除倾斜等问题，但其生成的边界框对应于预处理后的图像，而非提交给API的原图。目前已能恢复边界框的原始旋转角度，但无法针对GCP添加的裁剪操作完成正确的缩放/调整，导致边界框与原图文字未对齐，标题位置比预期低15%。

相关图像 #

• 原图：
• GCP预处理图像：
• 文本输出：

API返回的变换矩阵 #

[{'rows': 2, 'cols': 3, 'type': 6, 'data': 'r6uHAEEXyL/OYqWImW3vP2QKzkdFXH9AzmKliJlt77+vq4cAQRfIv5yIBglXqaZA'}]

解码倾斜角度的代码 #

rows = matrix_data[0]["rows"]
cols = matrix_data[0]["cols"]
data_type_code = matrix_data[0]["type"]
data_encoded = matrix_data[0]["data"]
data_binary = base64.b64decode(data_encoded)
matrix = np.frombuffer(data_binary, dtype=dtype).reshape((rows, cols))
rotation_radians = math.atan2(matrix[1, 0], matrix[0, 0])
skew_angle = math.degrees(rotation_radians)

边界框解析代码 #

box = Box(root=[Point(x=v.get("x", -1), y=v.get("y", -1)) for v in vertices])
box = box.skew_box(self.skew_angle)
box = box.scale_box(self.scale_x, self.scale_y)

比例计算代码 #

# 提交给API的原图比例
x_ratio = x_len / y_len if y_len != 0 else 1
y_ratio = y_len / x_len if x_len != 0 else 1
longest_side = "x" if x_len > y_len else "y"

# GCP预处理图像的比例
gcp_x_ratio = gcp_height / gcp_width if gcp_width != 0 else 1
gcp_y_ratio = gcp_width / gcp_height if gcp_height != 0 else 1
gcp_longest_side = "x" if gcp_width > gcp_height else "y"

if longest_side != gcp_longest_side:
    gcp_x_ratio, gcp_y_ratio = gcp_y_ratio, gcp_x_ratio

self.scale_x = x_ratio / gcp_x_ratio
self.scale_y = y_ratio / gcp_y_ratio

缩放计算实现代码 #

def scale_box(self, x: float, y: float, origin: Point = Point(x=0.5, y=0.5)):
    def scale_point_about_origin(point: Point, origin: Point) -> Point:
        translated_x = point.x - origin.x
        translated_y = point.y - origin.y
        scaled_x = translated_x * x
        scaled_y = translated_y * y
        return Point(x=scaled_x + origin.x, y=scaled_y + origin.y)

    return Box(root=[scale_point_about_origin(p, origin) for p in self.root])

边界框校正代码 #

def skew_box(self, angle: float, width: float = 1, height: float = 1):
    if angle == 0:
        return self

    theta = math.radians(-angle)
    cos_theta = np.cos(theta)
    sin_theta = np.sin(theta)
    origin_x = width / 2
    origin_y = height / 2

    def skew_point(point: Point) -> Point:
        x, y = point.x - origin_x, point.y - origin_y
        x_rotated = x * cos_theta - y * sin_theta
        y_rotated = x * sin_theta + y * cos_theta
        x_rotated, y_rotated = x_rotated + origin_x, y_rotated + origin_y

        return Point(x=x_rotated, y=y_rotated)

    return Box([skew_point(p) for p in self.root])

目前倾斜已校正，但边界框与原图文字未对齐，标题位置比预期低15%。

内容的提问来源于stack exchange，提问作者user19213041