如何在Pandas中对齐多组日期-收益列并填充缺失值?
问题描述
我的Pandas DataFrame存在观测值错位问题:多组日期列(如Date.0、Date.1)与对应收益列(如ABC Return、XYZ Return)是通过API单独生成的,导致日期无法匹配。
示例原始数据:
import pandas as pd import numpy as np data = {"Date.0": ["1/1/2022","1/2/2022", "1/3/2022","1/4/2022"], "ABC Return": [11, 21, 31, 41], "Date.1": ["1/1/2022","1/2/2022", "1/4/2022","1/5/2022"], "XYZ Return": [12, 22, 42, 51] } df = pd.DataFrame(data)
期望的目标格式(所有日期对齐,缺失值填充为NaN):
data = {"Date.0": ["1/1/2022","1/2/2022", "1/3/2022","1/4/2022","1/5/2022"], "ABC Return": [11, 21, 31, 41, np.NaN], "Date.1": ["1/1/2022","1/2/2022", "1/3/2022", "1/4/2022","1/5/2022"], "XYZ Return": [12, 22, np.NaN, 42, 51]} df = pd.DataFrame(data)
目前我有近60个这类变量,除了手动在Excel中合并外,还没找到可扩展的解决方案。
注:收益列仅以xxx Return结尾,无其他固定规律;日期列按Date.0、Date.1编号,有固定规律。
可行解决方案
基于Nick的方案调整后,以下代码可实现需求(已适配多变量场景):
import pandas as pd import numpy as np # 获取所有收益列的名称列表 ticker_list = [col for col in df.columns if col.endswith('Return')] # 收集所有日期列的唯一日期,构建基准日期索引 dates = pd.DataFrame(pd.concat([df[f'Date.{v}'] for v in range(len(ticker_list))]).unique()).set_index(0) date_counter = 0 dfs = [] for ticker in ticker_list: # 将当前收益列与对应日期列合并到基准日期索引上,填充缺失值为NaN dfs.append(dates.join(df[[f'Date.{date_counter}', ticker]].set_index(f'Date.{date_counter}')).fillna(np.NaN)) date_counter += 1 # 合并所有处理后的DataFrame,重置索引并命名为date out = pd.concat(dfs, axis=1).reset_index(names='date')
内容的提问来源于stack exchange,提问作者Tartaglia
相关产品推荐
相关产品推荐

