How a network computes, why gradients vanish, and what makes depth trainable
Everything painful in dl.24 came from one choice: which axis the mean and variance were taken over. Batch norm goes down the batch, so one example's output depends on its neighbours. Turn the statistics ninety degrees and that dependence is gone.
🔒 This is a Pro lesson — the interactive figure, worked examples, quiz and practice open with Pro access.
▶ Layer Norm and RMSNorm