Singular Value Decomposition (SVD)

Factorizing a matrix into rotation and scaling components.

∑Mathematical Definition

A = U Σ Vᵀ

For any m × n matrix A

U (Left Singular Vectors)

An m × m orthogonal matrix. Its columns are eigenvectors of AAᵀ.

Σ (Singular Values)

An m × n diagonal matrix with non-negative real numbers on the diagonal.

Vᵀ (Right Singular Vectors)

The transpose of an n × n orthogonal matrix V. Columns of V are eigenvectors of AᵀA.

Geometric Interpretation

SVD decomposes any linear transformation into three simple steps:

Rotation (Vᵀ): Rotates the input vector.
Scaling (Σ): Stretches or shrinks the vector along the coordinate axes.
Rotation (U): Rotates the result again.

Applications

•Image Compression: Approximating an image with a lower rank matrix.
•Dimensionality Reduction (PCA): Finding the most important features in data.
•Noise Reduction: Removing small singular values that correspond to noise.
•Pseudo-Inverse: Solving linear systems where A is not square.