如何将Pandas中的iterrows()转换为lambda函数？求更高效实现方案

原实现代码 #

你原本用iterrows()循环生成字典的代码如下：

%%time
match_ids_dict = {}

for index, row in df_match.iterrows():
    team1 = row['team1'] + ' Vs ' + row['team2']
    team2 = row['team2'] + ' Vs ' + row['team1']
    match_ids_dict[team1] = row['match_id']
    match_ids_dict[team2] = row['match_id']

match_ids_dict

用lambda替换iterrows()的方案 #

可以用apply()搭配lambda函数处理每行数据，生成包含两组键值对的结构，再展开成字典：

%%time
# 用lambda处理每行，输出两个键值对的元组
processed_rows = df_match.apply(
    lambda row: [(row['team1'] + ' Vs ' + row['team2'], row['match_id']),
                 (row['team2'] + ' Vs ' + row['team1'], row['match_id'])],
    axis=1
)
# 展开所有元组并转换为字典
match_ids_dict = dict(item for sublist in processed_rows for item in sublist)

更高效的向量化实现 #

上面的apply本质还是行级迭代，大数据量下推荐用向量化操作——底层是C实现，性能提升明显：

%%time
# 批量生成两种对战组合的字符串
pair1 = df_match['team1'] + ' Vs ' + df_match['team2']
pair2 = df_match['team2'] + ' Vs ' + df_match['team1']

# 合并所有键和对应的match_id（每个id对应两个键，所以重复一次）
all_keys = pd.concat([pair1, pair2])
all_values = pd.concat([df_match['match_id'], df_match['match_id']])

# 一键生成字典
match_ids_dict = dict(zip(all_keys, all_values))

或者更简洁的写法：

%%time
match_ids_dict = dict(zip(
    pd.concat([df_match['team1']+' Vs '+df_match['team2'], df_match['team2']+' Vs '+df_match['team1']]),
    pd.concat([df_match['match_id'], df_match['match_id']])
))

内容的提问来源于stack exchange，提问作者SHIVANI GUPTA