如何在Pandas/Numpy中对仅含有效值的序列计算滚动函数？

数据特征与需求 #

• 数据每行对应特定时间索引下某列的具体值
• 每行存在多个非NaN值，但数量有限
• 需要实现基于有效值、结合相对时间索引的梯度滚动函数

尝试过的无效方法 #

曾尝试Pandas、Numpy的常规处理方式，包括dropna方法、skipna参数，均无法满足需求。

示例代码与效果对比 #

生成测试数据 #

import numpy as np
import pandas as pd

fa = np.random.randn(10,4)
mask = np.zeros(40, dtype=bool)
mask[:15] = True
np.random.shuffle(mask)
mask = mask.reshape(10,4)
fa[mask] = np.nan

idx = pd.date_range("2023-01-01", periods=10, freq="S")
df = pd.DataFrame(fa, index=idx)

常规滚动求和的输出（不符合需求） #

执行df.rolling(3).apply(lambda s: s.sum())后得到：

0         1   2   3
2023-01-01 00:00:00       NaN       NaN NaN NaN
2023-01-01 00:00:01       NaN       NaN NaN NaN
2023-01-01 00:00:02       NaN -1.370696 NaN NaN
2023-01-01 00:00:03 -0.905364       NaN NaN NaN
2023-01-01 00:00:04 -1.207028       NaN NaN NaN
2023-01-01 00:00:05 -1.027719       NaN NaN NaN
2023-01-01 00:00:06 -0.080573 -1.087092 NaN NaN
2023-01-01 00:00:07 -0.069553       NaN NaN NaN
2023-01-01 00:00:08 -0.126387       NaN NaN NaN
2023-01-01 00:00:09 -0.126387       NaN NaN NaN

期望输出（忽略NaN，仅基于有效值计算） #

希望输出仅针对每列的有效值进行滚动计算，效果相当于对每列先执行df[n].dropna()再做滚动求和，之后手动填充回原索引：

2023-01-01 00:00:00       NaN       NaN       NaN       NaN
2023-01-01 00:00:01       NaN       NaN       NaN       NaN
2023-01-01 00:00:02       NaN -1.370696       NaN       NaN
2023-01-01 00:00:03 -0.589829 -1.289354       NaN       NaN
2023-01-01 00:00:04 -0.039064 -0.451169       NaN       NaN
2023-01-01 00:00:05 -0.398826 -1.087092       NaN  1.226619
2023-01-01 00:00:06  0.357316       NaN  0.131681  3.589662
2023-01-01 00:00:07 -0.028043       NaN       NaN       NaN
2023-01-01 00:00:08       NaN  0.161823 -0.746563       NaN
2023-01-01 00:00:09 -0.455660  0.213346       NaN  4.192798

关键难点 #

手动对每列单独处理并填充可以实现求和的示例效果，但实际需求的滚动函数需要结合时间索引作为输入，因此无法通过简单替换NaN为0的方式解决。

内容的提问来源于stack exchange，提问作者David Boshton