Hyperparameter Search

How models actually learn, from vanilla gradient descent to Adam

Hyperparameters are training choices set outside the learned parameters: learning rate, batch size, weight decay, warmup length, dropout, model width, and many others.

🔒 This is a Pro lesson — the interactive figure, worked examples, quiz and practice open with Pro access.

▶ Hyperparameter Search
← Initialization & Signal ScaleWhat Makes a Model Learn? →