How a network computes, why gradients vanish, and what makes depth trainable
dl.19 showed that the gradient reaching the first layer is a product of every Jacobian above it, and dl.20 showed what a per-layer factor below 1 does to that product. dl.21 and dl.22 pushed the factor toward 1 at step zero. Residual connections do something different: they change what a block computes, so that the product contains one term that no Jacobian multiplies.
🔒 This is a Pro lesson — the interactive figure, worked examples, quiz and practice open with Pro access.
▶ Residual Connections