Mixed Precision & Loss Scaling

How models actually learn, from vanilla gradient descent to Adam

Mixed precision training uses smaller number formats for speed and memory. Instead of storing every calculation in full precision (standard 32-bit floats), many operations use float16 or bfloat16: 16-bit formats that take half the memory in exchange for less precision and, for float16, a narrower range of representable sizes.

The risk is numerical range. Some gradients are tiny. If a tiny number is rounded to zero, the optimizer loses information. Loss scaling protects those small gradients by multiplying the loss before backpropagation, then dividing the gradients back down.

A kitchen scale that rounds to whole grams can miss a tiny pinch of spice. If you weigh ten identical pinches together, the scale can see the total. Then you divide by ten to recover one pinch. Loss scaling uses the same trick: make the small value easier to represent, then scale it back. The figure below is a reminder of what is at stake. Descent only works if each step's gradient survives the arithmetic; precision does not change the loop, it decides whether the tiny slopes near the minimum are still visible to it.

Where this lives in MLMixed precision is one reason large neural networks train fast on modern hardware. Optimizers still need the same concepts, but numerical scale becomes part of the training recipe.
▶ Mixed Precision & Loss Scaling
← Alternating Least SquaresBatch Size Scaling →