Pandas按条件批量替换多列值的高效实现方法
问题场景
目前我使用如下代码在满足特定条件时替换列取值:
df.loc[df['event'] == 1, 'pre'] = 0 df.loc[df['event'] == 1, 'post'] = 1 df.loc[df['event'] == 2, 'pre'] = 1 df.loc[df['event'] == 2, 'post'] = 0 df.loc[df['event'] == 4, 'pre'] = 1 df.loc[df['event'] == 4, 'post'] = 0
但该写法可扩展性差,无法适配更多条件场景,请问有没有更高效的实现方式?
完整测试代码如下:
import numpy as np import pandas as pd nfreq=500 arr=np.array([[11850,0,1], [12310,0,3], [13924,0,4], [16690,0,1], [17082,0,3], [18746,0,4], [21956,0,2], [22324,0,3], [23694,0,4], [25382,0,1], [25776,0,3], [28592,0,4], [31676,0,2], [32028,0,3], [33498,0,4]]) trange = np.where(arr == 3)[0] val=np.array([arr[trange, 0],arr[trange, 2], (arr[trange, 0]-arr[trange-1, 0])/nfreq, (arr[trange+1, 0]-arr[trange, 0])/nfreq]).T trange= np.where(arr[:,2] != 3)[0] val_oth=np.array([arr[trange, 0],arr[trange, 2],arr[trange, 2],arr[trange, 2]]).T val_oth[:,2]=1 val_oth[:,-1]=1 df = pd.DataFrame(np.vstack((val,val_oth)),columns=['timepoint','event','pre','post']) df.loc[df['event'] == 1, 'pre'] = 0 df.loc[df['event'] == 1, 'post'] = 1 df.loc[df['event'] == 2, 'pre'] = 1 df.loc[df['event'] == 2, 'post'] = 0 df.loc[df['event'] == 4, 'pre'] = 1 df.loc[df['event'] == 4, 'post'] = 0 df.sort_values(by='timepoint', ascending=True,inplace=True) df.reset_index(drop=True,inplace=True)
解决方案
你可以将事件匹配规则集中管理,用矢量化操作替代重复的.loc语句,大幅提升扩展性和执行效率,以下是几种常用实现方式:
方案1:映射字典循环赋值(最易维护,适合规则不多的场景)
把所有事件对应的pre、post取值统一存在字典里,新增规则仅需修改字典即可:
# 集中管理所有事件匹配规则,新增事件直接加键值对就行 event_rule = { 1: (0, 1), 2: (1, 0), 4: (1, 0) } for event, (pre_val, post_val) in event_rule.items(): df.loc[df['event'] == event, ['pre', 'post']] = pre_val, post_val
方案2:merge映射表(性能最优,适合事件规则非常多的场景)
将规则转成DataFrame后和原表合并,全程矢量化操作,数据量越大优势越明显:
rule_df = pd.DataFrame.from_dict( {1: [0,1], 2:[1,0], 4:[1,0]}, orient='index', columns=['pre_rule', 'post_rule'] ) # 合并规则直接覆盖原列 df = df.merge(rule_df, left_on='event', right_index=True, how='left') df['pre'] = df['pre_rule'] df['post'] = df['post_rule'] df.drop(columns=['pre_rule', 'post_rule'], inplace=True)
方案3:np.select多条件匹配(适合规则逻辑复杂,不是简单等值匹配的场景)
如果后续规则会加入大于、小于等复杂判断,用np.select更灵活:
conditions = [ df['event'] == 1, df['event'].isin([2,4]) ] pre_values = [0, 1] post_values = [1, 0] df['pre'] = np.select(conditions, pre_values, default=df['pre']) df['post'] = np.select(conditions, post_values, default=df['post'])
三种方案都能完全替代你原本的重复.loc写法,最终输出结果和原代码完全一致。
内容的提问来源于stack exchange,提问作者rpb
相关产品推荐
相关产品推荐

