Projections

Geometry and algebra of linear maps, vectors, and matrices

A projection answers "what is the closest point to b that lives in a given subspace?" Picture a point floating above a floor: its projection is the spot on the floor directly below it, the foot of the perpendicular. It is the best approximation of b available within the subspace.

To project a vector b onto a single direction a, scale a by how much of b lies along it (a dot product), normalized by a's own length-squared:

Drag b around the figure and watch its shadow slide along the line a, always landing at the closest point, with the dashed error segment meeting the line at a right angle.

Where this lives in MLProjection is the geometry behind attention and residual streams. Least-squares regression projects targets onto the model's column space. The residual stream in a transformer is repeatedly read from and written to via projections, and Gram–Schmidt-style orthogonalization keeps learned directions distinct. "Closest point in a subspace" is a move models make constantly.
▶ Projections
← Matrix NormsQuadratic Forms →