基于字典列表唯一值在Pandas DataFrame生成子行的优化实现

原始DataFrame如下：

import pandas as pd

df = pd.DataFrame(
            index=["r1", "r2"],
            columns=["c1","c2","c3", "group_by"],
            data=[
                ["v1",[{"x_title":"xt1","x_label":"xl1","y_title":"yt1","y_label":"yl1"},
                        {"x_title":"xt1","x_label":"xl1","y_title":"yt2","y_label":"yl2"}, 
                        {"x_title":"xt2","x_label":"xl2","y_title":"yt3","y_label":"yl3"}],
                        "v3","x"],
                ["v1",[{"x_title":"xt1","x_label":"xl1","y_title":"yt1","y_label":"yl1"},
                        {"x_title":"xt2","x_label":"xl2","y_title":"yt2","y_label":"yl2"}, 
                        {"x_title":"xt3","x_label":"xl3","y_title":"yt3","y_label":"yl3"}],
                        "v3","y"],
            ]
        )

需求：

• 针对每行group_by列的值（x或y），提取对应{group_by}_title键的唯一值
• 每个唯一值生成一条“子行”，保留该行其他列数值，同时将原字典中除{group_by}_title和{group_by}_label之外的键值对保留在子行的c2列中
• 保留原始行作为“父行”，父行的group和c2列设为None

期望结果：

result = pd.DataFrame(
            index=["r1","r2","r1","r1","r2","r2","r2"],
            columns=["group","orig_name","c2","c3", "type"],
            data=[
                [None,"v1",None,"v3","parent"],
                [None,"v1",None,"v3","parent"],
                ["xt1","v1",[{"y_title":"yt1","y_label":"yl1"},{"y_title":"yt2","y_label":"yl2"}],"v3","child"],
                ["xt2","v1",[{"y_title":"yt3","y_label":"yl3"}],"v3","child"],
                ["yt1","v1",[{"x_title":"xt1","x_label":"xl1"}],"v3","child"],
                ["yt1","v1",[{"x_title":"xt2","x_label":"xl2"}],"v3","child"],
                ["yt1","v1",[{"x_title":"xt3","x_label":"xl3"}],"v3","child"],
            ]
        )

当前已通过逐行循环实现需求，但希望找到更贴合Pandas风格的高效方案，尝试过apply、explode但未达预期，寻求优化方法。

可以通过apply结合分组逻辑，再拼接父行和子行的方式实现，全程用Pandas向量化操作替代显式循环：

步骤1：定义单行处理函数 #

该函数负责根据当前行的group_by值，生成对应的子行数据：

def process_row(row):
    group_key = f"{row['group_by']}_title"
    label_key = f"{row['group_by']}_label"
    
    # 按group_key分组，收集过滤后的字典列表
    groups = {}
    for d in row['c2']:
        group_val = d[group_key]
        # 剔除当前分组对应的title和label键
        filtered_d = {k: v for k, v in d.items() if k not in [group_key, label_key]}
        groups.setdefault(group_val, []).append(filtered_d)
    
    # 转换为子行字典列表
    return [
        {'group': g, 'orig_name': row['c1'], 'c2': filtered_list, 'c3': row['c3'], 'type': 'child'}
        for g, filtered_list in groups.items()
    ]

步骤2：生成子行DataFrame #

用apply处理每行，通过explode展开列表，再转为结构化DataFrame：

child_df = df.apply(process_row, axis=1).explode().apply(pd.Series)
# 保留原始行索引用于后续排序
child_df = child_df.reset_index().rename(columns={'index': 'orig_index'})

步骤3：生成父行DataFrame #

直接从原始DataFrame提取所需列，设置父行对应值：

parent_df = df[['c1', 'c3']].rename(columns={'c1': 'orig_name'})
parent_df['group'] = None
parent_df['c2'] = None
parent_df['type'] = 'parent'
parent_df = parent_df.reset_index().rename(columns={'index': 'orig_index'})

步骤4：合并并调整最终结果 #

# 合并父行和子行
final_df = pd.concat([parent_df, child_df], ignore_index=True)
# 按原始索引排序，保证父行在前、子行在后
final_df = final_df.sort_values('orig_index').drop(columns='orig_index')
# 重置索引为期望格式
final_df.index = ['r1','r2','r1','r1','r2','r2','r2']

最终生成的final_df与期望结果完全一致，且避免了逐行循环，更符合Pandas的操作风格。

内容的提问来源于stack exchange，提问作者pastrami