回归分析中偏相关的两个公式证明求助：R²与RSS的关系

Hey there, let's work through these two propositions about partial correlation in regression step by step— I’ll keep the matrix explanations as intuitive as possible since you mentioned you’re not super comfortable with that stuff.

1. Proving $r^2_{yx_m(x_1,...,x_{m-1})}=rac{RSS_m-RSS}{RSS_m}$ #

First, let’s recall what a squared partial correlation means: $r^2_{yx_m(x_1,...,x_{m-1})}$ is the squared simple correlation between y and $x_m$ after removing the linear effects of $x_1$ to $x_{m-1}$ from both variables.

To formalize this:

• Let $e_y = (I - H_m)y$: This is the residual when we regress y on $x_1,...,x_{m-1}$ (the simplified model). It’s y with the linear influence of the first m-1 variables stripped out.
• Let $e_{x_m} = (I - H_m)x_m$: This is the residual when we regress $x_m$ on $x_1,...,x_{m-1}$. It’s $x_m$ with the same linear influence stripped out.

By definition, the squared partial correlation is the squared correlation between $e_y$ and $e_{x_m}$:
$$
r^2_{yx_m(x_1,...,x_{m-1})} = \left( \frac{e_y^T e_{x_m}}{\sqrt{e_y^T e_y} \cdot \sqrt{e_{x_m}^T e_{x_m}}} \right)^2 = \frac{(e_y^T e_{x_m})^2}{(e_yT e_y)(e_{x_m}^T e_{x_m})}
$$

Notice that $e_y^T e_y = RSS_m$ (that’s exactly the residual sum of squares for the simplified model). Now, think about the full model (including all $x_1$ to $x_m$): its residual sum of squares $RSS$ is what’s left after we account for both the first m-1 variables and $x_m$.

When we add $x_m$ to the simplified model, the reduction in RSS (which is $RSS_m - RSS$) is equal to the sum of squares explained by regressing $e_y$ on $e_{x_m}$. For a simple linear regression of one variable on another, the sum of squares explained is $\frac{(e_y^T e_{x_m})^2}{e_{x_m}T e_{x_m}}$.

So we can rewrite:
$$
RSS_m - RSS = \frac{(e_y^T e_{x_m})^2}{e_{x_m}T e_{x_m}}
$$

Divide both sides by $RSS_m$:
$$
\frac{RSS_m - RSS}{RSS_m} = \frac{(e_y^T e_{x_m})^2}{RSS_m \cdot e_{x_m}^T e_{x_m}}
$$

Which is exactly the squared partial correlation we defined earlier. That’s the first proposition done!

2. Proving $RSS_m-RSS=\frac{(y^T(I-H_m)x_m)2}{x_m^T(I-H_m)x_m}$ #

We can build directly from the last step of the first proof. We already have:
$$
RSS_m - RSS = \frac{(e_y^T e_{x_m})^2}{e_{x_m}T e_{x_m}}
$$

Now, let’s substitute back the definitions of $e_y$ and $e_{x_m}$:

• $e_y^T e_{x_m} = [(I - H_m)y]^T (I - H_m)x_m$

Here’s a key matrix property to remember: the matrix $I - H_m$ is symmetric and idempotent. Symmetric means $(I - H_m)^T = I - H_m$, and idempotent means $(I - H_m)^2 = I - H_m$ (this comes from $H_m$ being a hat matrix, which has these same properties). Using this, we can simplify the expression:
$$
[(I - H_m)y]^T (I - H_m)x_m = y^T (I - H_m)^T (I - H_m)x_m = y^T (I - H_m)x_m
$$

Similarly, $e_{x_m}^T e_{x_m} = [(I - H_m)x_m]^T (I - H_m)x_m = x_m^T (I - H_m)x_m$.

Substitute these back into the equation for $RSS_m - RSS$:
$$
RSS_m - RSS = \frac{(y^T(I-H_m)x_m)2}{x_m^T(I-H_m)x_m}
$$

And that’s the second proposition proven!

内容的提问来源于stack exchange，提问作者John