对比DataFrame多列对，生成含差异_y列名的新列

给定数据与需求 #

示例数据 #

import pandas as pd

data = [{'name': 'Muhammad', 'age_x': 20, 'city_x': 'Karachi', 'age_y': 20, 'city_y': 'Karachi'},
        {'name': 'Ali', 'age_x': 19, 'city_x': 'Lahore', 'age_y': 30, 'city_y': None},
        {'name': 'Ahmed', 'age_x': 25, 'city_x': 'Islamabad', 'age_y': None, 'city_y': 'Islamabad'}]
            
df = pd.DataFrame(data)
cols = {'age_x':'age_y', 'city_x':'city_y'}

需求说明 #

根据cols字典定义的列对（如age_x与age_y、city_x与city_y），批量对比每组列的取值：

• 当列对值不同时，记录对应的_y后缀列名
• 将所有差异列名汇总到新的diff列中，无差异则显示None

预期输出 #

name     age_x  city_x    age_y city_y    diff
0   Muhammad 20     Karachi   20.0  Karachi   None
1   Ali      19     Lahore    30.0  None      age_y, city_y
2   Ahmed    25     Islamabad NaN   Islamabad age_y

实现代码 #

# 初始化diff列为空列表
df['diff'] = [[] for _ in range(len(df))]

# 遍历每个列对进行对比
for col_x, col_y in cols.items():
    # 构建差异掩码：排除值相等或两者都为缺失值的情况
    diff_mask = ~((df[col_x] == df[col_y]) | (pd.isna(df[col_x]) & pd.isna(df[col_y])))
    # 给有差异的行添加对应的_y列名
    df.loc[diff_mask, 'diff'] = df.loc[diff_mask, 'diff'].apply(lambda lst: lst + [col_y])

# 将列表格式的差异列转为逗号分隔字符串，空列表转为None
df['diff'] = df['diff'].apply(lambda x: ', '.join(x) if x else None)

运行结果 #

执行代码后得到的DataFrame与预期完全一致：

name  age_x     city_x  age_y     city_y            diff
0  Muhammad     20    Karachi   20.0    Karachi             None
1       Ali     19      Lahore   30.0       None  age_y, city_y
2     Ahmed     25  Islamabad    NaN  Islamabad          age_y

内容的提问来源于stack exchange，提问作者nzskra