How models actually learn, from vanilla gradient descent to Adam
RMSProp adapts the step size separately for each parameter. It tracks a moving average of squared gradients, then divides the gradient by the root of that average.
The effect is simple: coordinates with consistently large gradients get smaller effective steps; coordinates with small gradients get relatively larger steps. This helps when gradient scales differ wildly.
Imagine factory conveyor belts carrying packages of different weights. If every belt moves at the same raw motor command, heavy lines can lurch while light lines barely matter. RMSProp watches each belt's load and scales the command per belt. The figure below shows the geometry RMSProp is built to fight: a stretched bowl where one coordinate's gradients run consistently larger than the other's. RMSProp shrinks the steep coordinate's steps and, relatively, boosts the flat one's.