How models actually learn, from vanilla gradient descent to Adam
The loss landscape is the shape of L(θ) over parameter space. For neural networks it is high-dimensional and non-convex: it has broad flat stretches, sharply curved ones, saddle points that climb in some directions while dropping in others, and many separate low-loss regions that often turn out to be connected.
You cannot visualize the real landscape directly, but you can reason about local geometry: gradient, curvature, noise, and how different optimizers move through them.
A dune field after strong wind has broad flat patches, sharp ridges, and paths that look level from one direction but slope from another. A loss landscape has the same issue: local shape depends on direction. You can build the most important such shape in the figure below: slide the two curvatures until one is positive and the other negative. That is a saddle, level along one line and sloping along another, and it is the kind of stationary point that dominates high-dimensional landscapes.