探究时间序列分析与线性回归的关联、差异及扩展关系

Awesome question—this is exactly the kind of foundational comparison that helps build a rock-solid understanding of predictive modeling. Let’s dive into the connections, key differences (focused on model assumptions, as you asked), and whether time series analysis counts as an extension of linear regression.

Core Connections #

• Both fall under the umbrella of linear modeling: Many time series models (like the Autoregressive, AR, component of ARIMA) are essentially linear regressions that use lagged values of the target variable as predictors. At their core, both aim to fit linear relationships between variables.
• Both rely on Ordinary Least Squares (OLS) for parameter estimation: Linear regression uses OLS to minimize the squared error between predicted and actual values; AR models also use OLS to estimate coefficients for lagged terms.

Key Differences (Focused on Model Assumptions) #

This is where the two frameworks diverge most sharply. Let’s break down their core assumptions side by side:

Linear Regression Assumptions #

• Independent and Identically Distributed (i.i.d.) Errors: Residuals must be independent of each other and follow the same distribution (typically normal). No autocorrelation is allowed—if residuals are correlated over time, the model is invalid.
• Exogenous Predictors: Independent variables are assumed to be external (not dependent on the target variable or time) and fixed (non-random) across observations.
• No Multicollinearity: Predictors shouldn’t have strong linear relationships with each other.
• Fixed Data Structure: Observations are treated as independent (e.g., survey responses from different people, cross-sectional data).

Box-Jenkins Time Series Assumptions #

The Box-Jenkins framework (covering AR, MA, ARMA, ARIMA) is built specifically for time-dependent data, so its assumptions are tailored to that context:

• Stationarity: The time series must have a constant mean, variance, and autocovariance over time. If the series is non-stationary (e.g., has a trend or seasonality), you first apply differencing (the "I" in ARIMA) to make it stationary.
• Autocorrelation is Expected: Unlike linear regression, time series models explicitly leverage autocorrelation—they assume the current value of the series depends on past observations. This is the core pattern the model is designed to capture.
• Residuals are White Noise: After fitting the model, the remaining residuals must be white noise (mean 0, constant variance, no autocorrelation). This ensures no unmodeled patterns are left in the data.
• Endogenous Predictors: Predictors are lagged values of the target variable itself (endogenous), not external variables. Some extensions (like ARIMAX) include exogenous predictors, but the core Box-Jenkins models focus on the series’ own history.

Is Time Series Analysis an Extension of Linear Regression? #

Short answer: Some time series models are extensions of linear regression, but time series analysis as a field is a distinct, specialized branch of statistics focused on time-dependent data.

• For example, an AR(1) model is literally a linear regression where the predictor is the target variable’s previous value: y_t = β₀ + β₁y_{t-1} + ε_t. In this case, it’s a direct extension tailored to time-ordered data.
• However, time series analysis introduces entirely new concepts (stationarity, differencing, seasonal decomposition, ACF/PACF plots) that have no parallel in standard linear regression. These concepts are essential to handling the unique challenges of time-dependent data, which linear regression wasn’t designed to address.

To put it in context:

• Use linear regression when you want to explain how external variables affect a target (e.g., how advertising spend and price impact sales).
• Use Box-Jenkins models when you want to predict future values of a series based on its own past behavior (e.g., forecasting monthly website traffic, daily stock prices).

内容的提问来源于stack exchange，提问作者Renel Chesak