Moore-Penrose Pseudo-Inverse
Generalizing the matrix inverse for non-square matrices.
†Definition & Formula
A⁺ = V Σ⁺ Uᵀ
Computed using SVD components
The pseudo-inverse A⁺ exists for any matrix, unlike the regular inverse which only exists for non-singular square matrices.
How to compute Σ⁺?
Take the reciprocal of each non-zero singular value in Σ, and transpose the resulting matrix. Zero singular values remain zero.
Key Properties
A A⁺ A = A
A⁺ acts like a weak inverse.
A⁺ A A⁺ = A⁺
(A A⁺)ᵀ = A A⁺
A A⁺ is symmetric.
(A⁺ A)ᵀ = A⁺ A
A⁺ A is symmetric.
Applications
The most common use is solving Linear Least Squares problems.
Given a system Ax = b (where A is not square):
The solution that minimizes the error ||Ax - b||² is given by:
x = A⁺b