pandas执行resample重采样时如何保留BG_TS等全部列
问题原因
你调用resample('5T').sum()时,pandas默认仅对数值类型列执行求和聚合,BG_TS(datetime类型)、bias(字符串类型)这类非数值列会被直接丢弃,这是列丢失的核心原因。
另外你当前场景是补全5分钟间隔的时间序列,不需要对值做求和聚合,用sum()本身也不符合需求——会把同一时间窗口的SG值加总,和原始采样逻辑不符。
解决方案
直接用asfreq()方法完成重采样,该方法会保留所有原始列,缺失的时间点自动填充NaN,不需要额外做0替换。同时建议在筛选子集时加.copy(),避免触发pandas的链式赋值警告。
修正后的完整代码如下:
import pandas as pd import numpy as np BG_test_df = pd.DataFrame( {'PERSON_ID': [1, 1, 1], 'TS': ['2021-08-14 19:00:27', '2021-08-14 20:00:27', '2021-08-14 22:35:27'], 'bias': ["Not outside of acceptable operation. Refer to patient education","Not outside of acceptable operation. Refer to patient education","Suboptimal"]} ) CGM_test_df = pd.DataFrame( {'PERSON_ID': [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1], 'SG': [50, 51, 52, 53, 54, 55, 400, 400, 400, 400, 400, 400, 400, 400,50, 51, 52, 53, 54, 55, 400], 'TS': ['2021-08-14 18:30:27','2021-08-14 18:35:27','2021-08-14 18:40:27','2021-08-14 18:45:27','2021-08-14 18:50:27','2021-08-14 18:55:27', '2021-08-14 19:00:27', '2021-08-14 19:30:27','2021-08-14 19:35:27','2021-08-14 19:40:27','2021-08-14 19:45:27','2021-08-14 19:50:27','2021-08-14 19:55:27','2021-08-14 20:00:27', '2021-08-14 20:30:27','2021-08-14 20:35:27','2021-08-14 20:40:27','2021-08-14 20:45:27','2021-08-14 20:50:27','2021-08-14 20:55:27','2021-08-14 21:00:27'] } ) problematic = BG_test_df.loc[BG_test_df['bias'] == "Suboptimal"].copy() # 转换时间格式 problematic['BG_TS'] = pd.to_datetime(problematic['TS']) CGM_test_df['CGM_TS'] = pd.to_datetime(CGM_test_df['TS']) merged = CGM_test_df.merge(problematic, on = "PERSON_ID") # 5分钟间隔重采样,保留所有列 filled = (merged .set_index('CGM_TS') .resample('5T') .asfreq() .reset_index() )
可选:自定义聚合规则
如果你后续确实需要对不同列做不同聚合计算(比如SG求和、BG_TS取窗口内第一个值),可以用agg()方法逐列指定聚合逻辑,写法如下:
filled = (merged .set_index('CGM_TS') .resample('5T') .agg({ 'PERSON_ID': 'first', 'SG': 'sum', 'TS_x': 'first', # merge后CGM侧的TS会被命名为TS_x 'bias': 'first', 'BG_TS': 'first', 'TS_y': 'first' # merge后BG侧的TS会被命名为TS_y }) .reset_index() )
内容的提问来源于stack exchange,提问作者Dr Wampa
相关产品推荐
相关产品推荐

