How models actually learn, from vanilla gradient descent to Adam
The learning rate is the step size of gradient descent. The gradient says which way the loss rises fastest; the optimizer moves the opposite way. The learning rate η decides how far it moves.
Too small and training crawls. Too large and the update can jump over the useful region, bounce, or explode. Most optimizer problems that look mysterious are really step-size problems first.
A kayak shows the tradeoff. Tiny paddle strokes keep control but make progress slow. Huge strokes can turn the kayak past the channel, and then you waste energy correcting. The learning rate is the stroke length. Try it below: pick a start, then raise η between runs and watch steady progress turn into overshooting and bouncing.