statsmodels中bk_filter返回大量NaN，cf_filter正常的原因及解决方法

问题描述 #

将statsmodels库中的bk_filter和cf_filter封装到TimeSeriesFilter类后，调用样本数据时，BK滤波结果出现大量NaN值，但CF滤波结果完全正常。

相关代码 #

import pandas as pd
import numpy as np
import statsmodels.api as sm
from pandas import Timestamp


class TimeSeriesFilter:
    def __init__(self, data: pd.Series):
        self.data = data

    def bk_filter(self, low: float, high: float, K: int) -> pd.Series:
        filtered_data = sm.tsa.filters.bkfilter(self.data, low=low, high=high, K=K)
        return pd.Series(filtered_data, index=self.data.index)


    def cf_filter(self, low: float, high: float) -> pd.Series:
        filtered_data, _ = sm.tsa.filters.cffilter(self.data, low, high)
        return pd.Series(filtered_data, index=self.data.index)


if __name__ == "__main__":
    # Example usage
    data_dict = {'date': {0: Timestamp('2009-07-31 00:00:00'),
      1: Timestamp('2009-08-31 00:00:00'),
      2: Timestamp('2009-09-30 00:00:00'),
      3: Timestamp('2009-10-31 00:00:00'),
      4: Timestamp('2009-11-30 00:00:00'),
      5: Timestamp('2009-12-31 00:00:00'),
      6: Timestamp('2010-01-31 00:00:00'),
      7: Timestamp('2010-02-28 00:00:00'),
      8: Timestamp('2010-03-31 00:00:00'),
      9: Timestamp('2010-04-30 00:00:00'),
      10: Timestamp('2010-05-31 00:00:00'),
      11: Timestamp('2010-06-30 00:00:00'),
      12: Timestamp('2010-07-31 00:00:00'),
      13: Timestamp('2010-08-31 00:00:00'),
      14: Timestamp('2010-09-30 00:00:00'),
      15: Timestamp('2010-10-31 00:00:00'),
      16: Timestamp('2010-11-30 00:00:00'),
      17: Timestamp('2010-12-31 00:00:00'),
      18: Timestamp('2011-01-31 00:00:00'),
      19: Timestamp('2011-02-28 00:00:00'),
      20: Timestamp('2011-03-31 00:00:00'),
      21: Timestamp('2011-04-30 00:00:00'),
      22: Timestamp('2011-05-31 00:00:00'),
      23: Timestamp('2011-06-30 00:00:00'),
      24: Timestamp('2011-07-31 00:00:00'),
      25: Timestamp('2011-08-31 00:00:00'),
      26: Timestamp('2011-09-30 00:00:00'),
      27: Timestamp('2011-10-31 00:00:00'),
      28: Timestamp('2011-11-30 00:00:00'),
      29: Timestamp('2011-12-31 00:00:00'),
      30: Timestamp('2012-01-31 00:00:00'),
      31: Timestamp('2012-02-29 00:00:00'),
      32: Timestamp('2012-03-31 00:00:00'),
      33: Timestamp('2012-04-30 00:00:00'),
      34: Timestamp('2012-05-31 00:00:00'),
      35: Timestamp('2012-06-30 00:00:00'),
      36: Timestamp('2012-07-31 00:00:00')},
     'M0': {0: 34239.3,
      1: 34406.62,
      2: 36787.89,
      3: 35730.23,
      4: 36343.86,
      5: 38246.97,
      6: 40758.58,
      7: 42865.79,
      8: 39080.58,
      9: 39657.54,
      10: 38652.97,
      11: 38904.85,
      12: 39543.16,
      13: 39922.76,
      14: 41854.41,
      15: 41646.21,
      16: 42252.16,
      17: 44628.17,
      18: 58063.94,
      19: 47270.24,
      20: 44845.22,
      21: 45489.03,
      22: 44602.83,
      23: 44477.8,
      24: 45183.1,
      25: 45775.29,
      26: 47145.29,
      27: 46579.39,
      28: 47317.26,
      29: 50748.46,
      30: 59820.72,
      31: 51448.78,
      32: 49595.74,
      33: 50199.32,
      34: 49039.72,
      35: 49284.64,
      36: 49705.85}}
    
    data = pd.DataFrame.from_dict(data_dict)
    data.set_index('date', inplace=True)
    data = data.squeeze()
    ts_filter = TimeSeriesFilter(data)
    bk_filtered_data = ts_filter.bk_filter(low=2, high=24, K=12)
    cf_filtered_data = ts_filter.cf_filter(low=6, high=24)

输出结果 #

BK滤波结果 #

bk_filtered_data
Out[34]: 
date
2009-07-31            NaN
2009-08-31            NaN
2009-09-30            NaN
2009-10-31            NaN
2009-11-30            NaN
2009-12-31            NaN
2010-01-31            NaN
2010-02-28            NaN
2010-03-31            NaN
2010-04-30            NaN
2010-05-31            NaN
2010-06-30            NaN
2010-07-31   -2159.989063
2010-08-31   -2492.524398
2010-09-30   -2260.018439
2010-10-31   -1001.685999
2010-11-30    1328.383812
2010-12-31    3875.940765
2011-01-31    5152.155843
2011-02-28    4241.233283
2011-03-31    1822.874041
2011-04-30    -644.881046
2011-05-31   -2086.347557
2011-06-30   -2370.419937
2011-07-31   -2139.886330
2011-08-31            NaN
2011-09-30            NaN
2011-10-31            NaN
2011-11-30            NaN
2011-12-31            NaN
2012-01-31            NaN
2012-02-29            NaN
2012-03-31            NaN
2012-04-30            NaN
2012-05-31            NaN
2012-06-30            NaN
2012-07-31            NaN
Name: M0_cycle, dtype: float64

CF滤波结果 #

cf_filtered_data
Out[35]: 
date
2009-07-31    -183.665874
2009-08-31    -820.188908
2009-09-30   -1124.902975
2009-10-31    -602.228968
2009-11-30     623.045342
2009-12-31    1843.435748
2010-01-31    2310.802995
2010-02-28    1809.138889
2010-03-31     753.691320
2010-04-30    -247.494003
2010-05-31    -931.157993
2010-06-30   -1492.858574
2010-07-31   -2179.370052
2010-08-31   -2794.841015
2010-09-30   -2676.450777
2010-10-31   -1259.780188
2010-11-30    1244.448950
2010-12-31    3726.592050
2011-01-31    4831.139555
2011-02-28    3915.254495
2011-03-31    1525.608637
2011-04-30   -1017.616293
2011-05-31   -2586.387447
2011-06-30   -2954.155048
2011-07-31   -2689.806540
2011-08-31   -2412.524710
2011-09-30   -2141.868563
2011-10-31   -1354.132116
2011-11-30     328.850289
2011-12-31    2504.327607
2012-01-31    4064.411463
2012-02-29    3992.993040
2012-03-31    2216.544675
2012-04-30    -245.080611
2012-05-31   -1992.429074
2012-06-30   -2248.219212
2012-07-31   -1333.168637
Name: M0_cycle, dtype: float64

补充对比示例 #

import statsmodels.api as sm
import pandas as pd
dta = sm.datasets.macrodata.load_pandas().data
index = pd.date_range(start='1959Q1', end='2009Q4', freq='Q')
dta.set_index(index, inplace=True)
cycles = sm.tsa.filters.bkfilter(dta[['realinv']], 6, 24, 12)
print(cycles)

输出：

realinv_cycle
1962-03-31      12.005783
1962-06-30       7.675060
1962-09-30      -2.441555
1962-12-31      -9.325617
1963-03-31      -9.023360
                  ...
2005-09-30      56.278133
2005-12-31      83.304872
2006-03-31     101.048653
2006-06-30     105.254000
2006-09-30      93.629199

[179 rows x 1 columns]

原因分析 #

1. BK滤波的窗口限制 #

BK滤波是对称移动平均滤波，计算每个点的滤波值时，需要当前点前后各K个数据点。比如你设置K=12，意味着：

• 前12个数据点没有足够的历史数据支撑计算，后12个数据点没有足够的未来数据支撑计算，因此这些位置会返回NaN。
• 你的样本共37条数据，有效计算范围是第13到第25个数据点（对应2010-07-31至2011-07-31），与输出的非NaN区间完全吻合。

2. CF滤波的递归特性 #