Linear Combinations & Span

Geometry and algebra of linear maps, vectors, and matrices

Give yourself a few vectors and two moves: scale each one (multiply by any number) and add the results. Any vector you can build this way is a linear combination of your starting set. The complete collection of everything reachable is called the span.

Span is the central idea here, so picture it concretely. One nonzero vector, scaled every which way, sweeps out a line through the origin. Two vectors that point in genuinely different directions sweep out a whole plane. Add a third that pokes out of that plane and you fill all of 3-D space.

Stock your blender with two base ingredients — say a banana arrow and a berry arrow. A smoothie is any mix where you scale each base (more or less of it) and pour them together; that is a linear combination. The full menu of every smoothie you could possibly blend from those bases is their span — and if both bases pull in genuinely different directions, that menu fills the whole plane of flavours.

Where this lives in MLSpan is exactly "what a layer can express." A linear layer Wx can only produce outputs in the span of W's columns, its column space. If that span misses a direction your data needs, no choice of input can recover it; the layer is structurally blind to that direction. Picking architectures with enough width is, in part, making sure the reachable span is big enough.
▶ Linear Combinations & Span
← NormsLinear Independence & Basis →