How models actually learn, from vanilla gradient descent to Adam
A fixed learning rate is rarely best for a whole training run. Early training can handle larger moves because the parameters are far from useful settings. Later training often needs smaller moves to settle.
A schedule changes η over time. Warmup starts with a small learning rate and increases it gradually before the main schedule begins.
When you launch a kite, you do not yank the string to full tension instantly. You let it rise, feel the wind, then adjust the line as it stabilizes. Warmup is the gentle launch. The later schedule is how you manage the line after the kite is flying.