关于自相关影响最小二乘估计量的数学解释问询

Great question—let’s break down exactly why autocorrelation distorts the variance of OLS estimators, using concrete math to make the mechanism clear.

1. Baseline: Classic OLS Assumptions & Variance Formula #

First, let’s recap the core setup of the classical linear regression model (CLRM):

• We assume error terms are serially uncorrelated: ( E(\varepsilon_i \varepsilon_j) = 0 ) for all ( i \neq j )
• Error terms are homoscedastic: ( Var(\varepsilon_i) = \sigma^2 ) for all ( i )

Under these assumptions, the variance of the OLS estimator ( \hat{\beta} = (X'X)^{-1}X'y ) is:
[ Var(\hat{\beta}) = \sigma^2 (X'X)^{-1} ]
This is the formula used to calculate standard errors in basic regression output.

2. Autocorruption Breaks the Covariance Structure #

When autocorrelation exists (violating the serial independence assumption), the error covariance matrix ( \Omega = E(\varepsilon \varepsilon') ) is no longer a simple diagonal matrix ( \sigma^2 I ). For example, with first-order autocorrelation (( \varepsilon_t = \rho \varepsilon_{t-1} + u_t ), where ( |\rho| < 1 ) and ( u_t ) is white noise), ( \Omega ) has:

• Diagonal elements: ( \frac{\sigma^2}{1 - \rho^2} )
• Off-diagonal elements (for lag ( k )): ( \frac{\sigma^2 \rho^k}{1 - \rho^2} )

The true variance of ( \hat{\beta} ) now becomes:
[ Var(\hat{\beta}) = (X'X)^{-1} X' \Omega X (X'X)^{-1} ]
This is the key formula that reveals how autocorrelation warps our variance estimates.

3. Why Underestimation/Overestimation Happens #

The bias comes from comparing the OLS-estimated variance ( \sigma^2 (X'X)^{-1} ) to the true variance above. Let’s break down the two common scenarios:

Case 1: Positive Autocorrelation (( \rho > 0 )) #

When errors are positively correlated (e.g., today’s error is likely to match yesterday’s), the off-diagonal elements of ( \Omega ) are positive. For most time-series data (where ( X ) includes trends, lagged values, or slowly changing variables), ( X' \Omega X ) will have larger diagonal elements than ( \sigma^2 X'X ).

This means the true variance of ( \hat{\beta} ) is larger than the OLS estimate. We end up with standard errors that are too small—underestimating uncertainty, inflating t-statistics, and leading to false positive significance tests.

Case 2: Negative Autocorrelation (( \rho < 0 )) #

Negative autocorrelation (errors flip sign from one period to the next) makes the off-diagonal elements of ( \Omega ) negative. Here, ( X' \Omega X ) can be smaller than ( \sigma^2 X'X ), so the true variance is smaller than the OLS estimate. Our standard errors are too large, leading to underpowered tests that miss true significant effects.

4. Concrete Univariate Example #

Let’s use a simple univariate regression ( Y_t = \beta_0 + \beta_1 X_t + \varepsilon_t ) where ( X_t = t ) (a linear time trend) and first-order autocorrelation applies.

The OLS estimate of ( Var(\hat{\beta}_1) ) is:
[ \hat{Var}(\hat{\beta}_1) = \frac{\hat{\sigma}^{2}{\sum_{t=1}}n (X_t - \bar{X})^2} ]

The true variance (accounting for ( \rho )) simplifies to:
[ Var(\hat{\beta}1) = \frac{\hat{\sigma}^{2}{\sum_{t=1}}n (X_t - \bar{X})^2} \times \left( 1 + \frac{2\rho \sum{t=1}^{n-1} (X_t - \bar{X})(X_{t+1} - \bar{X})}{\sum_{t=1}^n (X_t - \bar{X})^2} \right) ]

Since ( X_t = t ), the cross-term ( \sum (X_t - \bar{X})(X_{t+1} - \bar{X}) ) is positive. So:

• If ( \rho > 0 ): The multiplier is greater than 1 → true variance > OLS estimate (variance underestimation)
• If ( \rho < 0 ): The multiplier is less than 1 → true variance < OLS estimate (variance overestimation)

This example makes the mathematical bias tangible.

Key Takeaway #

Autocorrelation doesn’t bias the OLS estimator itself (it remains unbiased!), but it biases the variance estimate of the estimator. The direction of the bias (under/overestimation) depends on the sign of the autocorrelation and the structure of your predictor variables ( X ).

内容的提问来源于stack exchange，提问作者KuDo