Gaussian Elimination

Geometry and algebra of linear maps, vectors, and matrices

Gaussian elimination is the systematic algorithm for solving Ax = b by hand or by machine. The idea: use simple row operations to carve the system into a staircase (echelon) form, then read the answer off by working backwards.

Three row operations are allowed, and none of them changes the solution set: swap two rows, multiply a row by a nonzero number, or add a multiple of one row to another. You wield them to knock entries down to zero, one column at a time.

The leading nonzero entry in each row is a pivot. Work top-down, using each pivot to clear everything beneath it, until the matrix is upper-triangular. Then back-substitute: the last row gives one variable directly; plug it into the row above and climb up.

Where this lives in MLGaussian elimination is the computational ancestor of LU decomposition, the routine your linear-algebra library actually calls to solve systems and invert matrices fast. You rarely run it by hand in ML, but it underlies the solvers behind closed-form regression, covariance computations, and any "solve this linear system" step inside a larger algorithm.
▶ Gaussian Elimination
← Ax = b: GeometryRank, Nullspace, Column Space →