Rank, Nullspace, Column Space

Geometry and algebra of linear maps, vectors, and matrices

Three quantities capture what a matrix really does. The column space is everything Ax can reach: the span of the columns, the matrix's "output region." The rank is the dimension of that column space, the number of genuinely independent directions A produces. And the null space is everything A crushes to zero, all x with Ax = 0.

Imagine giving directions using landmarks. If you say "go toward the tower" and "go toward the tower's twin right beside it," you have really only given one genuine direction — the second adds nothing new. The rank counts how many of a matrix's directions are truly independent like this; any direction that collapses to no movement at all belongs to the null space.

The dimensions obey a clean balance, the rank–nullity theorem: the input dimensions split into the directions that survive (rank) and the directions that get crushed (nullity).

Where this lives in MLRank measures a layer's true expressivity. A weight matrix that is low-rank has redundant neurons (several computing combinations of the others) and can be compressed with no loss. This is the engine of LoRA: replace a big weight update with a low-rank product BA, training far fewer parameters because the useful update lives in only a few directions.
▶ Rank, Nullspace, Column Space
← Gaussian EliminationMatrix Inverse →