Transpose

Geometry and algebra of linear maps, vectors, and matrices

The transpose Aᵀ flips a matrix across its main diagonal: rows become columns and columns become rows. Entry (i, j) swaps with entry (j, i). An (m×n) matrix becomes (n×m).

Imagine a spreadsheet where rows are people and columns are the months they each paid. Transposing it tips the whole table on its diagonal so rows become columns: now rows are months and columns are people. No number is lost or changed — each value just moves to its mirrored cell, where its row label and column label have traded places.

A matrix that equals its own transpose, A = Aᵀ, is symmetric: mirror-balanced across the diagonal, with Aᵢⱼ = Aⱼᵢ. These matrices are special enough that two whole later lessons are devoted to them.

Where this lives in MLTranspose is everywhere in backprop. The forward pass multiplies by W; the backward pass multiplies the incoming gradient by Wᵀ to send it to the previous layer. Attention scores are QKᵀ. And the Hessian and covariance matrices are symmetric (A = Aᵀ) by construction, which is exactly what guarantees the nice eigen-structure later lessons rely on.
▶ Transpose
← Matrix MultiplicationSpecial Matrices →