How models actually learn, from vanilla gradient descent to Adam
Adam combines two ideas: momentum for the average gradient, and RMSProp-style scaling for the average squared gradient. It then corrects early bias because those moving averages start at zero.
That combination makes Adam a common first choice in deep learning, especially when gradients are noisy and parameters have very different scales.
An autopilot can use two instruments at once. One shows the plane's average direction of drift. The other shows how turbulent that direction has been. Adam uses the same idea: move with the persistent drift, but scale the move by recent turbulence. In the figure you can watch the momentum half of Adam directly: raise β and the ricochet fades. Adam layers one more trick on top, dividing each coordinate's step by that coordinate's own recent gradient scale.