Pandas中iterrows转向量化操作：HC值跨DataFrame转移优化需求

核心思路 #

• 精准筛选捐赠方目标数据：通过指定的donor_group和donor_channel过滤donor_df，锁定需要扣减hc的行
• 计算单份额摊值：将总捐赠量donating_hc平均分配到每一行
• 原地扣减捐赠方hc：直接对筛选出的捐赠行执行hc扣减操作
• 映射并更新接收方数据：将捐赠行的wg和site映射到接收方的receiver_group和receiver_channel，合并到receiver_df自动处理新增行，再执行hc添加操作

向量化实现代码 #

import pandas as pd

def allocate_hc(donor_df, receiver_df, donor_group, donor_channel, receiver_group, receiver_channel, donating_hc):
    # 1. 筛选捐赠方目标行（布尔索引实现向量化筛选）
    donor_mask = (donor_df['group'] == donor_group) & (donor_df['channel'] == donor_channel)
    donor_subset = donor_df[donor_mask].copy()
    
    if donor_subset.empty:
        return  # 无捐赠行直接返回
    
    # 2. 计算单份额摊值
    per_row_allocation = donating_hc / len(donor_subset)
    
    # 3. 原地扣减捐赠方hc列
    donor_df.loc[donor_mask, 'hc'] -= per_row_allocation
    
    # 4. 构造接收方待添加数据
    receiver_data = donor_subset[['wg', 'site']].copy()
    receiver_data['group'] = receiver_group
    receiver_data['channel'] = receiver_channel
    receiver_data['hc'] = per_row_allocation
    
    # 5. 合并到接收方，自动处理新增行，原地更新hc值
    temp_index = ['group', 'channel', 'wg', 'site']
    receiver_df_temp = receiver_df.set_index(temp_index)
    receiver_data_temp = receiver_data.set_index(temp_index)
    
    # 相加合并，不存在的行自动新增
    receiver_df_temp = receiver_df_temp.add(receiver_data_temp, fill_value=0).reset_index()
    
    # 原地替换receiver_df数据
    receiver_df.drop(receiver_df.index, inplace=True)
    receiver_df[receiver_df_temp.columns] = receiver_df_temp

示例验证 #

示例1：双方行完全匹配 #

假设初始数据：

# 捐赠方数据
donor_data = [
    ['A', 'C1', 'wg1', 'site1', 10],
    ['A', 'C1', 'wg1', 'site2', 10],
    ['A', 'C1', 'wg2', 'site3', 10]
]
donor_df = pd.DataFrame(donor_data, columns=['group', 'channel', 'wg', 'site', 'hc']).set_index(['group', 'channel', 'wg'])

# 接收方数据
receiver_data = [
    ['B', 'C1', 'wg1', 'site1', 5],
    ['B', 'C1', 'wg1', 'site2', 5],
    ['B', 'C1', 'wg2', 'site3', 5]
]
receiver_df = pd.DataFrame(receiver_data, columns=['group', 'channel', 'wg', 'site', 'hc']).set_index(['group', 'channel', 'wg'])

执行allocate_hc(donor_df, receiver_df, 'A', 'C1', 'B', 'C1', 10)后：

• donor_df的3行hc各扣减10/3≈3.3333，结果为6.6667
• receiver_df的3行hc各增加3.3333，结果为8.3333

示例2：接收方存在缺失行 #

假设接收方初始缺少wg2-site3行：

receiver_data = [
    ['B', 'C1', 'wg1', 'site1', 5],
    ['B', 'C1', 'wg1', 'site2', 5]
]
receiver_df = pd.DataFrame(receiver_data, columns=['group', 'channel', 'wg', 'site', 'hc']).set_index(['group', 'channel', 'wg'])

执行相同分配操作后：

• receiver_df会自动新增['B', 'C1', 'wg2', 'site3']行，hc值为3.3333，另外两行hc更新为8.3333

关键说明 #

• 全程使用布尔索引、loc赋值、add合并等向量化操作，避免循环，大幅提升效率
• 通过设置临时多索引实现精准匹配，add方法的fill_value=0自动处理接收方缺失行的新增
• 最后通过清空原DataFrame并重新赋值实现原地修改，满足需求

内容的提问来源于stack exchange，提问作者sai naveen