SARIMAX处理5分钟高频时间序列数据的性能问题及替代方案咨询

我最近一直在用statsmodels包里的SARIMAX来建模时间序列数据，目的是理解数据的季节性和非平稳性。我的数据是5分钟间隔的高频数据，每天有288个观测值，但过程中遇到了不小的挑战：

• 性能问题：即使我配置了正确的参数（比如设置日季节性周期为288），SARIMAX的运行速度慢得离谱，有时候还会因为内存或计算资源不足直接崩溃。
• 模型适配性疑问：我怀疑SARIMAX可能并不适合高频数据集，这是真的吗？如果是的话，为什么SARIMAX在处理这类数据时效率特别低？

修复方案或替代模型（重要说明） #

注意：我完全不想对数据进行重采样或降采样，因为这样会丢失对我的分析至关重要的信息。

我的核心问题 #

• 有没有什么技术可以在不丢失信息的前提下，优化SARIMAX处理高频数据的性能？
• 或者，有没有专门为高频数据设计的更好的模型，同时还能让我理解季节性、趋势等时间维度的特征？

我之前写了一个可复现的示例代码，用来复现这个问题：

# !pip install pmdarima
# Import additional required libraries
import pandas as pd
import numpy as np
import pmdarima as pm
import matplotlib.pyplot as plt
from statsmodels.tsa.statespace.sarimax import SARIMAX
from sklearn.metrics import mean_absolute_error, mean_squared_error
from datetime import datetime, timedelta


# Generate timestamps for 30 consecutive days with 5-minute intervals
start_time = datetime.now().replace(hour=0, minute=0, second=0, microsecond=0)
timestamps = [start_time + timedelta(minutes=5 * i) for i in range(30 * 24 * 12)]  # 5-min epochs

# Generate 'avgcpu' data with seasonality
np.random.seed(42)  # For reproducibility
base_pattern = np.sin(np.linspace(0, 2 * np.pi, len(timestamps) // 30)) * 7.5 + 27.5
avgcpu = np.tile(base_pattern, 30) + np.random.normal(0, 2, len(timestamps))
avgcpu = np.clip(avgcpu, 20, 35)  # Normalize within the range [20, 35]

# Create a DataFrame
data = pd.DataFrame({
    "timestamp": timestamps,
    "avgcpu": avgcpu
}).sort_values("timestamp")


# Regenerate data and split into training and testing sets
test_size  = 2 * 24 * 12  # 2 days of data (288 * 2 observations)
train_data = data.iloc[:-test_size]
test_data  = data.iloc[-test_size:]

train_series = train_data["avgcpu"]
test_series  = test_data["avgcpu"]

我尝试用pm.auto_arima()和直接调用SARIMAX()两种方式，但都遇到了性能问题：

# Model 1: Using pm.auto_arima()
seasonal_period = 288  # Daily seasonality
model1 = pm.auto_arima(
    train_series,
    seasonal=True,
    m=seasonal_period,       # Seasonal period
    trace=True,              # Prints the model selection process
    error_action="ignore",   # Ignores non-converging models
    suppress_warnings=True,  # Suppresses warnings
    stepwise=True            # Stepwise model selection for faster computation
)

# Model 2: Using SARIMAX()
seasonal_period = 288  # Daily seasonality
model2 = SARIMAX(
    train_series,
    order=(1, 1, 1),                            # Simplified initial setup for ARIMA terms
    seasonal_order=(1, 1, 1, seasonal_period),  # Seasonal ARIMA terms
    enforce_stationarity=False,
    enforce_invertibility=False
)

# Fit SARIMAX model
sarimax_result = model2.fit(disp=False)

# Summary of models
print("Model 1 (auto_arima):")
print(model1.summary())

print("\nModel 2 (SARIMAX):")
print(sarimax_result.summary())

# Forecasting with both models for the test set period
forecast_model1, conf_int_model1 = model1.predict(n_periods=test_size, return_conf_int=True)
forecast_model2      = sarimax_result.get_forecast(steps=test_size)
forecast_mean_model2 = forecast_model2.predicted_mean
forecast_ci_model2   = forecast_model2.conf_int()

# Calculate metrics for both models
mae_model1 = mean_absolute_error(test_series, forecast_model1)
mse_model1 = mean_squared_error(test_series, forecast_model1)

mae_model2 = mean_absolute_error(test_series, forecast_mean_model2)
mse_model2 = mean_squared_error(test_series, forecast_mean_model2)

备注：内容来源于stack exchange，提问作者Mario