How a network computes, why gradients vanish, and what makes depth trainable
Every backward pass so far has been per-example. calc2.24's three gradients treat one example at a time and the batch only appears at the end, as a sum. Batch normalization is the first layer in this course where that is false. μ_B and σ_B are computed from the whole batch, so example 3's output is a function of example 1's input, and the derivative has to say so.
🔒 This is a Pro lesson — the interactive figure, worked examples, quiz and practice open with Pro access.
▶ The Gradient Through Batch Norm