如何高效处理Pandas DataFrame中特定Placement序列的条件计算?
问题描述
原始DataFrame如下:
Placement Value Order 0 high 10 1 1 med 5 2 2 high 9 3 3 low 3 4 4 med 7 5 5 low 2 6 6 med 6 7 7 high 9 8 8 med 4 9 9 low 2 10 10 high 8 11 11 med 6 12 12 high 8 13 13 med 5 14 14 low 1 15
需求:
检测以下四种Placement序列:
high -> lowlow -> highhigh -> med -> lowlow -> med -> high
对符合条件的序列执行:
- 计算序列中high与low的Value差值(high值减去low值),填入序列最后一行的新列
Diff - 新增
Measured From列,值为序列首行的Order - 删除
Diff和Measured From为空的行,得到目标DataFrame
原本打算通过迭代每行查看前两个Placement值实现,但希望找到Pandas中更高效的非迭代方法。
高效实现方案
用Pandas的向量化操作就能搞定,完全不用循环迭代,效率拉满,具体步骤如下:
1. 生成序列字符串,方便匹配目标模式
先把当前行和前1行、前2行的Placement值取出来,拼成序列字符串,这样就能快速判断是不是我们要找的模式:
# 获取前1行、前2行的Placement值 df['prev1'] = df['Placement'].shift(1) df['prev2'] = df['Placement'].shift(2) # 生成2步序列(前1行+当前行)和3步序列(前2行+前1行+当前行) df['seq_2'] = df['prev1'] + '->' + df['Placement'] df['seq_3'] = df['prev2'] + '->' + df['prev1'] + '->' + df['Placement']
2. 标记符合条件的行
定义目标序列集合,然后标记哪些行是目标序列的最后一行:
# 目标序列集合 target_seqs = {'high->low', 'low->high', 'high->med->low', 'low->med->high'} # 标记当前行是否是目标序列的最后一行 df['is_target'] = df['seq_2'].isin(target_seqs) | df['seq_3'].isin(target_seqs)
3. 计算Diff和Measured From列
根据序列的类型(2步或3步),分别计算差值和起始Order:
import numpy as np # 计算Diff列:分四种情况处理 df['Diff'] = np.where( df['seq_2'] == 'high->low', df['Value'].shift(1) - df['Value'], # high->low:前一行high值减当前low值 np.where( df['seq_2'] == 'low->high', df['Value'] - df['Value'].shift(1), # low->high:当前high值减前一行low值 np.where( df['seq_3'] == 'high->med->low', df['Value'].shift(2) - df['Value'], # high->med->low:前两行high值减当前low值 np.where( df['seq_3'] == 'low->med->high', df['Value'] - df['Value'].shift(2), # low->med->high:当前high值减前两行low值 np.nan ) ) ) ) # 计算Measured From列:序列首行的Order值 df['Measured From'] = np.where( df['seq_2'].isin(target_seqs), df['Order'].shift(1), # 2步序列取前一行的Order np.where( df['seq_3'].isin(target_seqs), df['Order'].shift(2), # 3步序列取前两行的Order np.nan ) )
4. 过滤并整理结果
删掉临时列和无值的行,保留需要的列:
# 过滤出有Diff值的行,保留目标列 result_df = df.dropna(subset=['Diff'])[['Placement', 'Value', 'Order', 'Diff', 'Measured From']] # 把数值转成整数(匹配目标输出格式) result_df['Diff'] = result_df['Diff'].astype(int) result_df['Measured From'] = result_df['Measured From'].astype(int)
最终得到的result_df就是目标DataFrame:
Placement Value Order Diff Measured From 3 low 3 4 -6 3 7 high 9 8 7 6 9 low 2 10 -7 8 10 high 8 11 6 10 14 low 1 15 -7 13
内容的提问来源于stack exchange,提问作者Jordan
相关产品推荐
相关产品推荐

