numpy.linalg.eigh与numpy.linalg.svd的算法、性能差异问询

Great question! Let's break down the differences between these two approaches clearly:

Algorithm & LAPACK Routine Differences #

First off: numpy.linalg.svd and the indirect SVD via numpy.linalg.eigh do NOT use the same algorithms or LAPACK routines.

• numpy.linalg.svd: By default, it calls LAPACK's gesdd (a divide-and-conquer SVD algorithm optimized for speed); older NumPy builds might fall back to gesvd (the traditional QR iteration-based SVD). It operates directly on the original matrix X—no need to precompute X'X or XX'.
• numpy.linalg.eigh: This is purpose-built for Hermitian matrices (real symmetric matrices are a special case). It uses LAPACK routines like syevd (for real symmetric matrices) or heevd (for complex Hermitian matrices), which are tailored for symmetric matrix eigendecomposition. To get SVD via this method, you first have to manually compute X.T @ X (real matrices) or X.conj().T @ X (complex matrices), then derive the SVD components from the eigenvalues and eigenvectors of that symmetric matrix.

Speed Differences #

• Direct svd is faster in most scenarios:
  The indirect approach adds an extra matrix multiplication step (computing X'X/XX'), which introduces significant overhead—especially for large X. The gesdd routine is purpose-built for SVD, so it avoids this extra computation and runs more efficiently overall.
• Indirect method may have speed advantages for extreme dimension imbalances:
  For example, if X is a tall-and-skinny matrix (rows m >> columns n), X'X is a small n×n matrix. Computing its eigendecomposition might be faster than running SVD on the much larger m×n X. But this is a niche case, and you'll trade off stability for this speed gain.

Stability Differences #

• Direct svd is significantly more stable:
  Computing X'X or XX' amplifies numerical errors. The eigenvalues of X'X are the squares of X's singular values—if X has a large condition number (e.g., near rank-deficient), small singular values squared will be swamped by the squares of large singular values, leading to severe loss of precision in the computed eigenvalues. This translates to inaccurate singular values and vectors when deriving SVD from them.
  The direct svd operates on the original matrix, avoiding this error amplification, so it's far more reliable for ill-conditioned matrices.
• The indirect method is only reasonably stable if X has a good condition number (small gap between singular values), but it still can't match the stability of direct SVD.

内容的提问来源于stack exchange，提问作者ArtificiallyIntelligent