Diagonalization

Geometry and algebra of linear maps, vectors, and matrices

Diagonalization rewrites a matrix in its own most natural coordinate system, the one built from its eigenvectors. In that system the matrix is diagonal: it does nothing but scale each eigen-axis by its eigenvalue. A tangled transformation becomes a simple one.

Here P has the eigenvectors as its columns and D is diagonal with the eigenvalues. Read the product right-to-left as a three-step recipe: P⁻¹ rotates into eigen-coordinates, D scales each axis, and P rotates back. A messy transformation, expressed as a pure stretch between two changes of view.

Diagonalization makes matrix powers almost free. Because the middle P⁻¹P pairs cancel, Aᵏ = P Dᵏ P⁻¹, and raising a diagonal matrix to a power just raises each diagonal entry to that power. No repeated matrix multiplication needed.

Where this lives in MLDiagonalization explains the long-run behavior of repeated linear maps, and almost every iterative algorithm is a repeated map near a fixed point. Whether training dynamics converge or blow up comes down to whether the relevant eigenvalues sit inside or outside the unit circle. The same idea, applied to symmetric matrices, becomes the spectral decomposition that powers PCA and the matrix…
▶ Diagonalization
← Eigenvectors & EigenvaluesSymmetric Matrices →