How a network computes, why gradients vanish, and what makes depth trainable
Take the hidden layer from dl.3 and delete the activation from it. The layer still holds a weight matrix and a bias, it still costs the same parameters, and it still runs. This lesson works out what such a stack computes, and the answer is the reason every hidden layer in every network ends in a φ.
🔒 This is a Pro lesson — the interactive figure, worked examples, quiz and practice open with Pro access.
▶ Why a Nonlinearity Is Required