Pandas resample:非精确时间戳重采样全为NaN的原因问询
问题描述
我有一个从设备获取的DataFrame,其timestamp为非精确秒级,示例数据如下:
hr timestamp 2022-11-02 20:23:20.850611 72 2022-11-02 20:23:21.868609 71 2022-11-02 20:23:22.932606 71 2022-11-02 20:23:24.057612 72 2022-11-02 20:23:25.182701 71 2022-11-02 20:23:25.932692 74 2022-11-02 20:23:27.057694 72 2022-11-02 20:23:28.182689 72 2022-11-02 20:23:28.932730 72 2022-11-02 20:23:30.057686 71 2022-11-02 20:23:31.182692 69 2022-11-02 20:23:31.932689 68 2022-11-02 20:23:33.057890 67 2022-11-02 20:23:34.182879 68 2022-11-02 20:23:34.932871 68 2022-11-02 20:23:36.057870 66 2022-11-02 20:23:37.182873 68 2022-11-02 20:23:37.933040 72 2022-11-02 20:23:39.058044 73 2022-11-02 20:23:40.183046 78 2022-11-02 20:23:40.959045 84 2022-11-02 20:23:42.058044 84 2022-11-02 20:23:43.183052 75 2022-11-02 20:23:43.936050 73 2022-11-02 20:23:45.058047 81
我尝试使用以下代码进行重采样,获取精确秒级、250ms分辨率的估计值:
df_resamp = df.resample('250ms').interpolate('cubic') print(df_resamp.head(30))
但返回结果全为NaN:
timestamp 2022-11-02 20:23:20.750 NaN 2022-11-02 20:23:21.000 NaN 2022-11-02 20:23:21.250 NaN 2022-11-02 20:23:21.500 NaN 2022-11-02 20:23:21.750 NaN 2022-11-02 20:23:22.000 NaN 2022-11-02 20:23:22.250 NaN 2022-11-02 20:23:22.500 NaN 2022-11-02 20:23:22.750 NaN 2022-11-02 20:23:23.000 NaN 2022-11-02 20:23:23.250 NaN 2022-11-02 20:23:23.500 NaN 2022-11-02 20:23:23.750 NaN 2022-11-02 20:23:24.000 NaN 2022-11-02 20:23:24.250 NaN 2022-11-02 20:23:24.500 NaN 2022-11-02 20:23:24.750 NaN 2022-11-02 20:23:25.000 NaN 2022-11-02 20:23:25.250 NaN 2022-11-02 20:23:25.500 NaN 2022-11-02 20:23:25.750 NaN 2022-11-02 20:23:26.000 NaN 2022-11-02 20:23:26.250 NaN 2022-11-02 20:23:26.500 NaN 2022-11-02 20:23:26.750 NaN 2022-11-02 20:23:27.000 NaN 2022-11-02 20:23:27.250 NaN 2022-11-02 20:23:27.500 NaN 2022-11-02 20:23:27.750 NaN 2022-11-02 20:23:28.000 NaN
请问为何会出现全NaN的情况?
问题原因
- 重采样锚点不匹配:
resample('250ms')默认以整点(如分钟的0秒)为锚点生成时间序列,你的原始数据起始时间是2022-11-02 20:23:20.850611,而重采样生成的第一个时间点是2022-11-02 20:23:20.750,早于原始数据起始时间,且后续所有重采样时间点和原始数据的时间戳完全无重叠,没有可用的非NaN值启动插值。 - 插值依赖已有数据:
interpolate必须基于已存在的非NaN值计算,若重采样后的序列中没有保留原始数据的时间点对应值,就无法进行插值计算。
解决方案
方式1:锚定重采样起始点为原始数据开头
让重采样序列从原始数据的第一个时间戳开始对齐,确保原始数据点被包含在内:
# 以原始数据起始时间为锚点重采样 df_resamp = df.resample('250ms', origin='start').interpolate('cubic') print(df_resamp.head(30))
方式2:先保留原始数据点再插值
先生成250ms间隔的空序列,填充原始数据值后再插值:
# 生成250ms间隔序列 df_resamp = df.resample('250ms').asfreq() # 将原始数据的值填充到对应时间点 df_resamp.loc[df.index, 'hr'] = df['hr'] # 执行三次样条插值 df_resamp = df_resamp.interpolate('cubic') print(df_resamp.head(30))
补充:强制从精确秒级开始
如果需要严格从整点秒(如2022-11-02 20:23:21.000)作为起始点,可以手动指定锚点:
from pandas import Timestamp # 指定精确秒作为重采样锚点 df_resamp = df.resample('250ms', origin=Timestamp('2022-11-02 20:23:21')).interpolate('cubic')
内容的提问来源于stack exchange,提问作者Jean Nobrega
相关产品推荐
相关产品推荐

