如何用EasyOCR合并垂直边界框并裁剪护照图像用于MRZ提取

• 持有原始护照图像及当前EasyOCR输出的带边界框图像，尝试通过EasyOCR的height_ths参数（浮点型，默认值0.5，控制合并时边界框的最大高度差值比例）合并最后两个垂直排列的边界框，但未成功。
• 当前仅能实现检测边界框并绘制的代码如下：

image = "/content/drive/MyDrive/DLD_Project/Passport_Data/100_Images_content/100_Images/1.jpg"
image = PIL.Image.open(image)
reader = easyocr.Reader(['en'])
bounds = reader.readtext('/content/drive/MyDrive/DLD_Project/Passport_Data/100_Images_content/100_Images/1.jpg')

from PIL import ImageDraw

def draw_boxes(image, bounds, color='yellow', width=2, box_size=3):
    draw = ImageDraw.Draw(image)
    for bound in bounds:
        p0, p1, p2, p3 = bound[0]
        # 扩大边界框尺寸
        p0 = (p0[0]-box_size, p0[1]-box_size)
        p1 = (p1[0]+box_size, p1[1]-box_size)
        p2 = (p2[0]+box_size, p2[1]+box_size)
        p3 = (p3[0]-box_size, p3[1]+box_size)
        draw.line([*p0, *p1, *p2, *p3, *p0], fill=color, width=width)
    return image

draw_boxes(image, bounds)

• 核心需求：实现垂直相邻边界框的合并，裁剪出对应区域后输入MRZ codechecker提取MRZ文本。

1. 调整EasyOCR参数优化自动合并 #

height_ths仅控制高度差异允许范围，垂直合并还受y_ths（控制垂直方向的间距/重叠阈值，默认0.5）影响，可同时调整两个参数：

# 修改readtext参数，调大允许的高度差和垂直间距阈值
bounds = reader.readtext(
    '/content/drive/MyDrive/DLD_Project/Passport_Data/100_Images_content/100_Images/1.jpg',
    height_ths=1.0,  # 允许100%的高度差
    y_ths=0.6        # 调大垂直方向的合并阈值
)

2. 手动合并垂直边界框 #

若参数调整仍无效，可手动筛选并合并符合条件的垂直框：

def merge_vertical_boxes(bounds, y_gap_threshold=20, height_diff_ratio=0.3):
    # 按边界框顶部y坐标排序，保证从上到下遍历
    sorted_bounds = sorted(bounds, key=lambda x: x[0][0][1])
    merged_bounds = []
    
    i = 0
    while i < len(sorted_bounds):
        curr_box = sorted_bounds[i]
        curr_p0, curr_p1, curr_p2, curr_p3 = curr_box[0]
        curr_top = curr_p0[1]
        curr_bottom = curr_p2[1]
        curr_height = curr_bottom - curr_top
        
        # 检查下一个框是否满足垂直合并条件
        merged = False
        if i + 1 < len(sorted_bounds):
            next_box = sorted_bounds[i+1]
            next_p0, _, _, next_p3 = next_box[0]
            next_top = next_p0[1]
            next_bottom = next_p3[1]
            next_height = next_bottom - next_top
            
            # 判断垂直间距是否过小，且高度差异在允许比例内
            vertical_gap = next_top - curr_bottom
            height_diff = abs(curr_height - next_height) / max(curr_height, next_height)
            
            if vertical_gap < y_gap_threshold and height_diff < height_diff_ratio:
                # 合并框：取最左、最上、最右、最下的坐标
                merged_p0 = (min(curr_p0[0], next_p0[0]), curr_top)
                merged_p1 = (max(curr_p1[0], next_p1[0]), curr_top)
                merged_p2 = (max(curr_p2[0], next_p2[0]), next_bottom)
                merged_p3 = (min(curr_p3[0], next_p3[0]), next_bottom)
                # 合并文本和置信度
                merged_text = f"{curr_box[1]} {next_box[1]}"
                merged_conf = (curr_box[2] + next_box[2]) / 2
                merged_bounds.append([[merged_p0, merged_p1, merged_p2, merged_p3], merged_text, merged_conf])
                i += 2  # 跳过已合并的下一个框
                merged = True
        
        if not merged:
            merged_bounds.append(curr_box)
            i += 1
    
    return merged_bounds

# 使用合并后的边界框
merged_bounds = merge_vertical_boxes(bounds)
draw_boxes(image, merged_bounds)

3. 裁剪MRZ区域并传入codechecker #

合并完成后，定位MRZ对应的边界框并裁剪图像：

# 假设最后一个合并后的框为MRZ区域（可根据文本特征/位置进一步判断）
mrz_box = merged_bounds[-1][0]
# 计算裁剪坐标：left, top, right, bottom
left = min(mrz_box[0][0], mrz_box[3][0])
top = min(mrz_box[0][1], mrz_box[1][1])
right = max(mrz_box[1][0], mrz_box[2][0])
bottom = max(mrz_box[2][1], mrz_box[3][1])

# 裁剪图像
mrz_cropped = image.crop((left, top, right, bottom))
# 保存或直接传入MRZ codechecker
mrz_cropped.save("/content/drive/MyDrive/DLD_Project/mrz_cropped.jpg")

内容的提问来源于stack exchange，提问作者Pravallika V