最小二乘问题求证：满秩矩阵A的伪逆2范数等于1/σᵣ

Alright, let's break this down step by step—singular value decomposition (SVD) is the perfect tool here since it directly links the Moore-Penrose inverse to a matrix's singular values.

• Start with the SVD of $A$
  Given $A \in \mathbb{R}^{m \times n}$ with $\text{rank}(A) = r$ and $A$ full-rank (so $r = \min(m,n)$), its singular value decomposition can be written as:
  $$A = U\Sigma V^T$$
  Where:

  • $U \in \mathbb{R}^{m \times m}$ and $V \in \mathbb{R}^{n \times n}$ are orthogonal matrices ($U^TU = I_m$, $V^TV = I_n$)
  • $\Sigma \in \mathbb{R}^{m \times n}$ is a diagonal matrix with singular values $\sigma_1 \geq \sigma_2 \geq \dots \geq \sigma_r > 0$ on its main diagonal; all other entries are 0.

• Write the Moore-Penrose inverse $A^+$ using SVD
  The Moore-Penrose inverse of $A$ has a straightforward form in terms of its SVD:
  $$A^+ = V\Sigma^+ U^T$$
  Here, $\Sigma^+ \in \mathbb{R}^{n \times m}$ is the pseudo-inverse of $\Sigma$. It's constructed by:

  • Taking the transpose of $\Sigma$
  • Replacing each non-zero singular value $\sigma_i$ with $1/\sigma_i$
  • Leaving all zero entries as 0

  So $\Sigma^+$ is a diagonal matrix with diagonal entries $1/\sigma_1, 1/\sigma_2, \dots, 1/\sigma_r$ (in order), and zeros elsewhere.

• Use 2-norm invariance under orthogonal transformations
  A key property of the 2-norm (spectral norm) is that it's invariant under multiplication by orthogonal matrices. That means for any matrix $B$ and orthogonal matrices $Q, P$, we have:
  $$||QBP||_2 = ||B||_2$$

  Applying this to $A^+ = V\Sigma^+ U^T$, since $V$ and $U^T$ are orthogonal:
  $$||A^+||_2 = ||V\Sigma^+ U^T||_2 = ||\Sigma^+||_2$$

• Compute the 2-norm of $\Sigma^+$
  The 2-norm of a diagonal matrix is equal to the absolute value of its largest diagonal entry. For $\Sigma^+$, our diagonal entries are $1/\sigma_1 \leq 1/\sigma_2 \leq \dots \leq 1/\sigma_r$ (since $\sigma_1 \geq \sigma_2 \geq \dots \geq \sigma_r > 0$).

  The largest entry here is $1/\sigma_r$, where $\sigma_r$ is the smallest positive singular value of $A$.

  By definition of the 2-norm:
  $$||\Sigma^+||_2 = \max\left{\frac{1}{\sigma_1}, \frac{1}{\sigma_2}, \dots, \frac{1}{\sigma_r}\right} = \frac{1}{\sigma_r}$$

Final Result #

Combining all the steps above, we conclude:
$$||A^+||_2 = \frac{1}{\sigma_r}$$

内容的提问来源于stack exchange，提问作者gohan