Geometry and algebra of linear maps, vectors, and matrices
Principal component analysis finds the directions in which your data varies most, then lets you describe each point with just a few of those directions instead of all the original features. It is the standard tool for dimensionality reduction, and under the hood it is SVD applied to your data.
The recipe is short. Center the data (subtract the mean so the cloud sits at the origin), take the SVD of the data matrix, and read off the answers: the principal components are the top singular directions, and each component's variance is its singular value squared (over n−1).
Picture a stretched, tilted cloud of points. The first principal component is the long axis of the cloud, the single direction capturing the most variance. The second is perpendicular to it, capturing the most of what's left, and so on. Project onto the first few and you keep the shape while shedding dimensions.