Eigenvectors & Eigenvalues

Geometry and algebra of linear maps, vectors, and matrices

Most vectors get knocked off course when a matrix acts on them: they rotate as well as stretch. But a few special directions are invariant. The matrix only stretches or flips them, never turns them. Those are the eigenvectors, and the stretch factor is the eigenvalue.

Read it aloud: applying A to its eigenvector v gives back the same direction, just scaled by λ. If λ = 2, that direction doubles; if λ = −1, it flips; if λ = 0.5, it shrinks by half. The eigenvectors form the skeleton of the transformation, the axes along which it acts most simply.

Drag a vector around the figure. Most directions visibly rotate under A; only along the eigenvector directions does the output stay parallel to the input.

Where this lives in MLEigenvectors are the directions a process naturally moves along. In PCA, the eigenvectors of the covariance matrix are the axes of greatest variance, the directions your data actually spreads. In optimization, the eigenvalues of the Hessian describe the curvature of the loss in each direction: large eigenvalues are steep walls, small ones are flat valleys, and their ratio (the condition number)…
▶ Eigenvectors & Eigenvalues
← Matrix InverseDiagonalization →