How a network computes, why gradients vanish, and what makes depth trainable
dl.17 ended with a debt. Reverse mode evaluates each layer's local derivative at that layer's forward values, and it reaches the bottom layer last, so every one of those forward values has to still be in memory when the backward sweep arrives. A framework records them as it goes, and the record has a name: the tape.
🔒 This is a Pro lesson — the interactive figure, worked examples, quiz and practice open with Pro access.
▶ The Tape: Memory, Recomputation and Checkpointing