How models actually learn, from vanilla gradient descent to Adam
Optimizer diagnostics means reading the training run before changing the training recipe. A loss curve, a gradient norm, and a validation curve usually tell you whether the problem is step size, data, scale, overfitting, or a real modeling limit.
This is not guesswork. Each failure pattern has a typical shape: exploding loss, flat loss, noisy but improving loss, training loss falling while validation loss rises, or sudden NaN values.
Aquarium test strips give a useful picture. You do not fix cloudy water by pouring in random chemicals. You first test pH, ammonia, and nitrate, then treat the reading that is actually bad. Optimizer diagnostics do the same for training: measure first, then change the thing the measurement points to. Use the figure to calibrate your eye. Run it once for a healthy descent, then crank η up and reproduce the unstable, bouncing pattern on demand.