Ax = b: Geometry

Geometry and algebra of linear maps, vectors, and matrices

The equation Ax = b is the central computation of linear algebra: given a transformation A and a target b, which input x lands on the target? Read it as geometry and the answer's character becomes visible before you compute anything.

There are two ways to picture it. The row picture: each equation is a line (in 2-D) or a plane (in 3-D), and the solution is where they all intersect. The column picture: b must be a linear combination of A's columns, and x holds the combination weights.

Geometrically there are exactly three cases. The lines cross at one point (a unique solution); they are parallel and distinct (no solution, the targets never meet); or they are the same line (infinitely many solutions). Drag the lines in the figure through all three.

Where this lives in MLReal ML systems are usually overdetermined: far more equations (data points) than unknowns (parameters), so an exact Ax = b almost never has a solution. That's the whole reason for least squares (a later lesson). When you can't hit b exactly, find the x that gets closest. Linear regression is precisely this "no exact solution, so minimize the miss" situation.
▶ Ax = b: Geometry
← Special MatricesGaussian Elimination →