使用Pandas itertuples构建矩阵:填充每行指定日期后首次事件日期
解决DataFrame每行填充指定日期后各类事件首次出现日期的问题
原代码的逻辑仅判断当前行事件类型是否匹配,直接填充当前行日期或'na',完全未实现当前行stamp日期之后和首次出现的核心需求,以下是修正后的实现方案:
高效实现方案(推荐)
利用日期预处理+二分查找提升效率,避免逐行遍历全量数据:
import pandas as pd from bisect import bisect_right # 初始化原始数据 id_list = [1,1,1,1,1,2,2,2,2,3,3,3,3,4,5,6,7,7,7,8,8,8,9,9,9,10,10,10] fact_list = ["IC", "AC","IC","AC","IC","AC", "CC", "CD","IC","CC", "CD","IC","AC", "CD","IC","AC", "CC", "CD","IC","AC", "CC", "CD","IC","AC","IC","IC","AC","IC"] stamp_list = ['1979-02-22','1973-11-06','1986-03-12','1986-01-24', '2012-05-22', '2009-01-18', '1992-01-14', '1985-06-05','2001-07-05','2008-11-19','2000-10-13','2002-04-18','1987-08-17','1977-04-09','1984-03-22','1994-08-08','2005-07-09','1982-05-03','2016-01-30','2019-03-10','1981-03-23','1979-07-21','2023-01-14','2018-06-23','1995-08-27','2020-11-08','2014-02-17','1977-09-08'] data = pd.DataFrame({"ID": id_list, "fact": fact_list, "stamp": stamp_list}) # 转换日期格式并排序,确保日期比较的正确性 data['stamp'] = pd.to_datetime(data['stamp']) data.sort_values(by="stamp", inplace=True) # 预处理:按事件类型分组,收集并排序所有对应日期 fact_dates = {} for fact_type in ['AC', 'CD', 'IC', 'CC']: dates = data[data['fact'] == fact_type]['stamp'].sort_values().tolist() fact_dates[fact_type] = dates # 定义函数:查找当前日期之后的首个事件日期 def get_first_future_date(current_stamp, date_list): idx = bisect_right(date_list, current_stamp) return date_list[idx] if idx < len(date_list) else pd.NaT # 为每行生成各事件类型的首次未来日期列 for fact_type in ['AC', 'CD', 'IC', 'CC']: data[fact_type] = data['stamp'].apply(lambda x: get_first_future_date(x, fact_dates[fact_type])) print(data)
逻辑说明
- 先将
stamp转为datetime类型,避免字符串比较的误差,同时对DataFrame按日期排序。 - 预处理阶段把每个事件类型的所有日期收集并排序,用
bisect_right快速定位第一个大于当前日期的位置,比逐行遍历全量数据效率提升显著。 - 对每行的日期,调用函数匹配各事件类型的首个未来日期,无符合条件的日期返回
pd.NaT(日期类型的缺失值)。
基于itertuples的修正版本
如果坚持使用itertuples遍历,可按以下方式修改:
import pandas as pd from bisect import bisect_right # 初始化并处理数据(同上方) id_list = [1,1,1,1,1,2,2,2,2,3,3,3,3,4,5,6,7,7,7,8,8,8,9,9,9,10,10,10] fact_list = ["IC", "AC","IC","AC","IC","AC", "CC", "CD","IC","CC", "CD","IC","AC", "CD","IC","AC", "CC", "CD","IC","AC", "CC", "CD","IC","AC","IC","IC","AC","IC"] stamp_list = ['1979-02-22','1973-11-06','1986-03-12','1986-01-24', '2012-05-22', '2009-01-18', '1992-01-14', '1985-06-05','2001-07-05','2008-11-19','2000-10-13','2002-04-18','1987-08-17','1977-04-09','1984-03-22','1994-08-08','2005-07-09','1982-05-03','2016-01-30','2019-03-10','1981-03-23','1979-07-21','2023-01-14','2018-06-23','1995-08-27','2020-11-08','2014-02-17','1977-09-08'] data = pd.DataFrame({"ID": id_list, "fact": fact_list, "stamp": stamp_list}) data['stamp'] = pd.to_datetime(data['stamp']) data.sort_values(by="stamp", inplace=True) # 准备结果结构 structure = {'ID': [], 'stamp':[], 'fact': [], 'AC':[], 'CD':[], 'IC':[], 'CC':[]} fact_types = ['AC', 'CD', 'IC', 'CC'] # 预处理每个事件的日期列表 fact_dates = {ft: data[data['fact'] == ft]['stamp'].sort_values().tolist() for ft in fact_types} for row in data.itertuples(): current_stamp = row.stamp # 填充基础字段 structure["ID"].append(row.ID) structure["stamp"].append(current_stamp) structure["fact"].append(row.fact) # 为每个事件类型查找首个未来日期 for ft in fact_types: dates = fact_dates[ft] idx = bisect_right(dates, current_stamp) structure[ft].append(dates[idx] if idx < len(dates) else pd.NaT) # 转为DataFrame result_df = pd.DataFrame(structure) print(result_df)
内容的提问来源于stack exchange,提问作者ZuZiTeK
相关产品推荐
相关产品推荐

