带边界框的结构化文本数据最优存储结构及转Pandas方案咨询

我有一个带关联边界框的结构化文本片段数据集，想找最优存储结构来高效存这些信息，还要能转成Pandas DataFrame方便后续操作。
每个类别可以包含多个值，每个值关联多个带边界框的文本片段，数据样例如下：

word    label           x1_pixel    y1_pixel    x3_pixel    y3_pixel
text_1  O                  1           2           3           4
text_2  O                  5           6           7           8
text_3  B-CLASS_NAME_1     9           10          11          12
text_4  I-CLASS_NAME_1     13          14          15          16
text_5  I-CLASS_NAME_1     17          18          19          20
text_6  O                  21          22          23          24
text_7  O                  25          26          27          28
text_8  B-CLASS_NAME_1     29          30          31          32
text_9  I-CLASS_NAME_1     33          34          35          36
text_10 I-CLASS_NAME_1     37          38          39          40
text_11 O                  41          42          43          44
text_12 O                  45          46          47          48

我现在设计的存储结构是这样的：

{
    "CLASS_NAME_1": [
        [
            ("text_3", "bounding_box"),
            ("text_4", "bounding_box"),
            ("text_5", "bounding_box"),
        ],
        [
            ("text_8", "bounding_box"),
            ("text_9", "bounding_box"),
            ("text_10", "bounding_box"),
        ]
    ],

    "CLASS_NAME_2" : [
        ...
    ],
    ...
}

本来打算写个类来处理存储逻辑，传类别名称就能创建新类别或追加到已有类别，代码如下：

class Key_Value_BB:
    class_names = {}
    prev_start = ""
    
    def __init__(self, class_name):
        self.class_name = class_name
        if class_name in self.class_names:
            self.append_to_existing(class_name)
        else:
            self.create_new(class_name)
    
    def append_to_existing(self, class_name):
        if(self.prev_start == 'I'):
            # append to the existing value
            pass
        self.prev_start = class_name[0]
        pass
    
    def create_new(self, class_name):
        pass

但搞不定把同批次的I标签文本片段存入同一个列表的逻辑。另外也想知道怎么处理数据差异（比如某个值只有B-CLASS_NAME_1、I-CLASS_NAME_1两个实例，不是三个），最终要实现对应文本和像素值的替换（比如用text_8替换text_3，text_9替换text_4这类操作）。

一、优化后的存储结构 #

你的初始思路没问题，但可以把bounding_box换成元组或字典，直接存取像素值，不用额外解析。推荐结构：

{
    "CLASS_NAME_1": [
        # 每个子列表对应一组连续的B+I标签片段
        [
            {"word": "text_3", "bbox": (9,10,11,12)},
            {"word": "text_4", "bbox": (13,14,15,16)},
            {"word": "text_5", "bbox": (17,18,19,20)}
        ],
        [
            {"word": "text_8", "bbox": (29,30,31,32)},
            {"word": "text_9", "bbox": (33,34,35,36)},
            {"word": "text_10", "bbox": (37,38,39,40)}
        ]
    ],
    "CLASS_NAME_2": [
        # 示例：只有B+I两个实例的情况
        [
            {"word": "text_x", "bbox": (x1,y1,x3,y3)},
            {"word": "text_y", "bbox": (x2,y2,x4,y4)}
        ]
    ]
}

用字典存每个片段的信息，可读性和操作性更强，不管一组有多少个B/I实例都能兼容。

二、处理B/I标签的逻辑实现 #

不用复杂的类，直接遍历原始数据（先转成DataFrame），按B标签分组，自动把后续同类别I标签归到同一组：

import pandas as pd

# 先把原始数据转成DataFrame
df = pd.DataFrame([
    ["text_1", "O", 1,2,3,4],
    ["text_2", "O",5,6,7,8],
    ["text_3", "B-CLASS_NAME_1",9,10,11,12],
    ["text_4", "I-CLASS_NAME_1",13,14,15,16],
    ["text_5", "I-CLASS_NAME_1",17,18,19,20],
    ["text_6", "O",21,22,23,24],
    ["text_7", "O",25,26,27,28],
    ["text_8", "B-CLASS_NAME_1",29,30,31,32],
    ["text_9", "I-CLASS_NAME_1",33,34,35,36],
    ["text_10", "I-CLASS_NAME_1",37,38,39,40],
    ["text_11", "O",41,42,43,44],
    ["text_12", "O",45,46,47,48]
], columns=["word", "label", "x1_pixel", "y1_pixel", "x3_pixel", "y3_pixel"])

# 初始化存储结构
storage = {}

# 遍历DataFrame处理标签
current_group = None
current_class = None

for _, row in df.iterrows():
    label = row["label"]
    if label == "O":
        # 遇到O标签，重置当前组
        current_group = None
        current_class = None
        continue
    
    # 拆分标签的前缀和类别
    prefix, cls = label.split("-", 1)
    bbox = (row["x1_pixel"], row["y1_pixel"], row["x3_pixel"], row["y3_pixel"])
    segment = {"word": row["word"], "bbox": bbox}
    
    if prefix == "B":
        # 新的实体组，创建新列表
        if cls not in storage:
            storage[cls] = []
        current_group = [segment]
        storage[cls].append(current_group)
        current_class = cls
    elif prefix == "I" and cls == current_class:
        # 同一类别的I标签，追加到当前组
        if current_group is not None:
            current_group.append(segment)

print(storage)

这段代码能自动处理任意长度的B+I组，不管是2个还是N个实例都能正确分组，完全兼容数据差异。

三、存储结构与DataFrame的互转 #

1. 从存储结构转回DataFrame #

如果需要把存储好的结构还原成原始格式的DataFrame：

rows = []
for cls, groups in storage.items():
    for group in groups:
        for idx, segment in enumerate(group):
            prefix = "B" if idx == 0 else "I"
            label = f"{prefix}-{cls}"
            x1, y1, x3, y3 = segment["bbox"]
            rows.append([segment["word"], label, x1, y1, x3, y3])
# 若需要补充O标签，可额外从原始数据中合并
result_df = pd.DataFrame(rows, columns=["word", "label", "x1_pixel", "y1_pixel", "x3_pixel", "y3_pixel"])

2. 直接基于DataFrame做替换操作 #

后续的替换等操作，直接在DataFrame上做更高效：

• 单个文本替换：

# 替换text_3为text_8，同步替换像素值
df.loc[df["word"] == "text_3", ["word", "x1_pixel", "y1_pixel", "x3_pixel", "y3_pixel"]] = ["text_8", 29,30,31,32]
# 替换text_4为text_9
df.loc[df["word"] == "text_4", ["word", "x1_pixel", "y1_pixel", "x3_pixel", "y3_pixel"]] = ["text_9",33,34,35,36]

• 整组替换（比如把第一组CLASS_NAME_1替换成第二组内容）：

# 定位第一组和第二组的行索引
group1_mask = (df["label"].str.startswith("B-CLASS_NAME_1")) | (df["label"].str.startswith("I-CLASS_NAME_1"))
group1_indices = df[group1_mask].iloc[:3].index
group2_values = df[group1_mask].iloc[3:6][["word", "x1_pixel", "y1_pixel", "x3_pixel", "y3_pixel"]].values

# 执行替换
df.loc[group1_indices, ["word", "x1_pixel", "y1_pixel", "x3_pixel", "y3_pixel"]] = group2_values

四、关键注意点 #

• 存储结构优先选字典嵌套列表+字典的形式，兼顾可读性和操作效率，比纯元组更直观。
• 处理I标签时，必须检查其类别是否和当前组一致，避免跨类别I标签被错误归组。
• 后续的替换、分析等操作，直接在Pandas DataFrame上执行效率更高，存储结构主要用于临时的实体分组存储。

内容的提问来源于stack exchange，提问作者scarecrow