Pandas datetime索引序列含缺失值时重采样求均值的实现问题咨询
问题根因
给已有固定索引的DataFrame直接赋值新列时,pandas会自动按现有索引做对齐匹配,仅保留当前索引范围内的对应值,索引外的数据会被直接丢弃,因此第一个序列的索引范围直接决定了最终DataFrame的长度,和后续序列的长度无关。
最优解决方案
不要逐列往空DataFrame中赋值,直接使用pandas原生的pd.concat()方法沿列维度合并所有重采样后的序列:
- 该方法会自动取所有输入序列索引的并集作为最终索引
- 无匹配值的位置自动填充为NaN,完全符合你缺失值标记的需求
- 后续你原本使用的
mean(axis=1, skipna=True)逻辑无需修改,会自动忽略NaN计算有效数据的均值
修改后的代码示例
针对你给出的测试代码,仅需替换最后合并部分的逻辑即可:
import pandas as pd ls = [] t1 = [ '00:00:00', '00:01:00', '00:02:00', '00:03:00', '00:04:00', '00:05:00', '00:06:00', '00:07:00', '00:08:00', '00:09:00', '00:10:00', '00:11:00', '00:12:00', '00:13:00', '00:14:00', '00:15:00'] val1 = [52, 52, 54, 54, 54, 58, 53, 52, 59, 61, 58, 60, 61, 58, 52, 54] t2 = ['00:00:00', '00:01:00', '00:02:00', '00:03:00', '00:04:00', '00:05:00', '00:06:00', '00:07:00', '00:08:00', '00:09:00', '00:10:00', '00:11:00', '00:12:00', '00:13:00', '00:14:00', '00:15:00', '00:16:00', '00:17:00', '00:18:00', '00:19:00', '00:20:00', '00:21:00', '00:22:00', '00:23:00', '00:24:00', '00:25:00', '00:26:00', '00:27:00', '00:28:00', '00:29:00', '00:30:00', '00:31:00', '00:32:00', '00:33:00', '00:34:00', '00:35:00', '00:36:00', '00:37:00', '00:38:00', '00:39:00', '00:40:00', '00:41:00', '00:42:00', '00:43:00', '00:44:00', '00:45:00', '00:46:00', '00:47:00', '00:48:00', '00:49:00', '00:50:00', '00:51:00', '00:52:00', '00:53:00', '00:54:00', '00:55:00', '00:56:00', '00:57:00', '00:58:00', '00:59:00', '01:00:00', '01:01:00', '01:02:00', '01:03:00', '01:04:00', '01:05:00', '01:06:00', '01:07:00', '01:08:00', '01:09:00', '01:10:00', '01:11:00', '01:12:00', '01:13:00', '01:14:00', '01:15:00', '01:16:00', '01:17:00', '01:18:00', '01:19:00', '01:20:00', '01:21:00', '01:22:00', '01:23:00', '01:24:00', '01:25:00', '01:26:00', '01:27:00', '01:28:00', '01:29:00', '01:30:00', '01:31:00', '01:32:00', '01:33:00', '01:34:00', '01:35:00', '01:36:00', '01:37:00', '01:38:00', '01:39:00'] val2 = [52, 52, 54, 54, 54, 58, 53, 52, 59, 61, 58, 60, 61, 58, 52, 54, 53, 54, 61, 53, 56, 56, 58, 56, 69, 69, 60, 53, 55, 53, 53, 53, 54, 56, 53, 53, 53, 52, 53, 53, 53, 54, 54, 54, 53, 53, 53, 54, 54, 54, 54, 54, 55, 55, 54, 55, 53, 54, 54, 55, 54, 53, 53, 53, 54, 54, 53, 53, 54, 54, 54, 55, 56, 53, 55, 52, 52, 51, 48, 49, 48, 48, 49, 49, 49, 50, 49, 48, 48, 48, 48, 49, 49, 48, 48, 49, 50, 49, 50, 49] df1 = pd.DataFrame({'values': val1, 'time': t1}) df1['time'] = pd.to_datetime(df1['time']) df1 = df1.set_index('time') ls.append(df1) df2 = pd.DataFrame({'values': val2, 'time': t2}) df2['time'] = pd.to_datetime(df2['time']) df2 = df2.set_index('time') ls.append(df2) # 核心修改:用concat沿列合并,自动取索引并集 dfnew = pd.concat(ls, axis=1) # 按你的需求重命名列即可 dfnew.columns = ['vals1', 'vals2'] # 计算均值逻辑不变 dfnew['means'] = dfnew.mean(axis=1, skipna=True) print(dfnew.shape) # 输出为(100, 3),符合预期
额外优化建议
如果你需要确保最终索引是完整的24小时每小时间隔(即使所有天的该时段都没有数据也保留索引),可以在重采样后补充调用asfreq方法:
df1 = df1.resample('1H').sum().asfreq('1H')
该操作会强制生成完整的1小时间隔索引,缺失数据自动填充为NaN,避免因所有天都缺失某时段导致索引漏项。
内容的提问来源于stack exchange,提问作者its_broke_again
相关产品推荐
相关产品推荐

