How models actually learn, from vanilla gradient descent to Adam
Optimization can fail before it starts if the initial scale is wrong. If weights are too small, signals and gradients can vanish. If weights are too large, activations and gradients can explode or saturate.
Initialization chooses a starting distribution for weights so signal size stays roughly stable as it moves through layers. Xavier and He initialization are two common rules.
Stage lighting has to start at the right level. Too dim and the actors disappear. Too bright and the scene washes out. Initialization sets the starting brightness of signals so each layer can pass useful information forward and backward. The figure below shows the core danger as pure math: a signal multiplied by roughly the same factor r at every layer is a geometric sequence. Slide r just below or above 1 and watch what many layers do to it: silence or explosion. Initialization exists to keep that factor near 1.