如何基于其他行的聚合值删除指定数据行?
数据处理需求与实现方案
需求说明
在同一Month和MainAggregate分组下,若所有**非SubAggregate='All'**行的ValueTraded总和等于该分组内SubAggregate='All'行的ValueTraded数值,则删除该SubAggregate='All'的行。
当前思路与已完成代码
思路逻辑:按MainAggregate和Month分组后,分组的ValueTraded总和 = 非All行总和 + All行数值。当非All行总和等于All行数值时,分组总和 = 2 * All行数值,此时需要删除对应的All行。目前仅完成分组求和步骤:
Tester = data.groupby(["Month", "MainAggregate"], as_index=False)["ValueTraded"].sum() Tester["ValueTraded"] = Tester["ValueTraded"] / 2
示例数据与期望输出
原始数据(DATA)
| Month | MainAggregate | SubAggregate | ValueTraded |
|---|---|---|---|
| 2023-01 | A | X | 50 |
| 2023-01 | A | Y | 50 |
| 2023-01 | A | All | 100 |
| 2023-01 | B | X | 30 |
| 2023-01 | B | All | 50 |
| 2023-02 | A | X | 40 |
| 2023-02 | A | All | 40 |
期望输出
| Month | MainAggregate | SubAggregate | ValueTraded |
|---|---|---|---|
| 2023-01 | A | X | 50 |
| 2023-01 | A | Y | 50 |
| 2023-01 | B | X | 30 |
| 2023-01 | B | All | 50 |
| 2023-02 | A | X | 40 |
完整实现代码
方案一:使用transform直接计算标记
通过transform在原数据中添加分组总和与All行数值,再构建过滤条件:
import pandas as pd # 计算每个分组的ValueTraded总和 group_total = data.groupby(["Month", "MainAggregate"])["ValueTraded"].transform("sum") # 提取每个分组中SubAggregate='All'的ValueTraded值 all_value = data.groupby(["Month", "MainAggregate"])["ValueTraded"].transform( lambda g: g[g.index.isin(data[data["SubAggregate"] == "All"].index)].iloc[0] if (g.index.isin(data[data["SubAggregate"] == "All"].index)).any() else 0 ) # 构建过滤掩码:非All行全部保留;All行仅当分组总和不等于2*All值时保留 keep_mask = (data["SubAggregate"] != "All") | (group_total != 2 * all_value) # 过滤得到结果 final_data = data[keep_mask].reset_index(drop=True)
方案二:合并All行数据后判断
先提取All行的参考值合并到原数据,再计算非All行总和进行判断:
import pandas as pd # 提取所有SubAggregate='All'的行,重命名列作为参考值 all_reference = data[data["SubAggregate"] == "All"].rename( columns={"ValueTraded": "All_Value"} )[["Month", "MainAggregate", "All_Value"]] # 将参考值合并到原数据 merged_data = pd.merge(data, all_reference, on=["Month", "MainAggregate"], how="left") # 计算每个分组中非All行的ValueTraded总和 non_all_sum = merged_data[merged_data["SubAggregate"] != "All"].groupby( ["Month", "MainAggregate"] )["ValueTraded"].transform("sum") # 构建过滤条件 keep_mask = (merged_data["SubAggregate"] != "All") | (non_all_sum != merged_data["All_Value"]) # 过滤并清理列 final_data = merged_data.drop(columns="All_Value")[keep_mask].reset_index(drop=True)
内容的提问来源于stack exchange,提问作者Marc225
相关产品推荐
相关产品推荐

