The mathematics of uncertainty
Suppose the truth is distribution p, but you encode outcomes using a different model q. Cross-entropy is the average surprise you actually pay: surprise measured by your model q, but averaged over how often events really occur under p:
🔒 This is a Pro lesson — the interactive figure, worked examples, quiz and practice open with Pro access.
▶ Cross-Entropy