AR(p)与ARIMA模型滚动预测气温时出现时序偏移问题咨询

Hey there, let's break down why your rolling forecasts with AR(p) and ARIMA are showing that frustrating time shift relative to actual temperature observations. I’ve dealt with this exact problem before, so here are the most common culprits and how to fix them:

1. Misaligned Prediction Timestamps & Rolling Window Logic #

This is the #1 reason for time shifts in rolling forecasts. When you train on data up to time t, you need to predict time t+1—and make sure that prediction’s timestamp lines up perfectly with the test set’s t+1 observation.

Fix: #

Double-check how you’re generating predictions and updating your training set. Here’s a corrected code snippet using StatsModels:

import pandas as pd
from statsmodels.tsa.arima.model import ARIMA

# Assume `ts` is your full temperature time series with a proper DatetimeIndex
n_train = 365  # Adjust based on your initial training window size
train = ts.iloc[:n_train].copy()
test = ts.iloc[n_train:]
predictions = []

for idx in range(len(test)):
    # Train model (use ARIMA(p,0,0) for pure AR(p))
    model = ARIMA(train, order=(5, 1, 0))  # Adjust order based on your data
    results = model.fit()
    
    # Predict the NEXT time step (exactly the test set's current index)
    pred = results.get_prediction(start=len(train), end=len(train))
    pred_value = pred.predicted_mean.iloc[0]
    predictions.append(pred_value)
    
    # Update training set with the REAL test observation (not the prediction!)
    train = pd.concat([train, test.iloc[[idx]]])

Key point: start=len(train) targets the timestamp immediately after your current training window, which matches the test set’s current row.

2. Invalid or Missing Time Series Index #

If your data doesn’t have a continuous, correctly formatted DatetimeIndex, your predictions and actual values will never line up properly. This is easy to overlook!

Fix: #

Ensure your time series has a valid index:

# Convert index to DatetimeIndex if it isn't already
ts.index = pd.to_datetime(ts.index)

# Resample to fix missing time steps (e.g., hourly data)
ts = ts.asfreq('H')  # Replace 'H' with your data's frequency (D for daily, M for monthly)

After this, verify your index with print(ts.index)—it should show a continuous sequence of dates/times.

3. Poorly Chosen Model Order (p/d/q) #

If your AR(p) order is too low, the model can’t capture enough of the temporal dependencies in temperature data, leading to lagged predictions. If it’s too high, overfitting can cause weird shifts too. For temperature data, don’t forget about seasonality—regular ARIMA won’t handle daily/yearly cycles well.

Fixes: #

• Select AR(p) order: Use the Partial Autocorrelation Function (PACF) to pick a reasonable p:

  from statsmodels.tsa.stattools import pacf
  import matplotlib.pyplot as plt
  import numpy as np
  
  # Compute PACF for differenced data (if using non-stationary data)
  pacf_vals = pacf(ts.diff().dropna(), nlags=20)
  plt.plot(pacf_vals)
  plt.axhline(y=-1.96/np.sqrt(len(ts)), linestyle='--', color='gray')
  plt.axhline(y=1.96/np.sqrt(len(ts)), linestyle='--', color='gray')
  plt.title("PACF Plot for AR Order Selection")
  plt.show()
  

  Pick the first lag where the PACF value crosses above the upper confidence interval.

• Add seasonality: If your temp data has daily/yearly cycles, switch to SARIMA (Seasonal ARIMA):

  from statsmodels.tsa.statespace.sarimax import SARIMAX
  
  # Example for hourly data with daily seasonality (s=24)
  model = SARIMAX(train, order=(5,1,0), seasonal_order=(1,1,0,24))
  results = model.fit()
  

4. Validate Alignment with Visualization #

Once you’ve adjusted your code, confirm alignment by plotting predictions and actual values together:

pred_df = pd.DataFrame({
    'Actual Temperature': test.values,
    'Predicted Temperature': predictions
}, index=test.index)

pred_df.plot(figsize=(12, 6))
plt.title("Temperature Forecast vs Actual")
plt.xlabel("Time")
plt.ylabel("Temperature")
plt.show()

If the lines are still shifted, double-check your rolling window update logic—you might be accidentally adding predictions to the training set instead of real values.

内容的提问来源于stack exchange，提问作者Henry