Exponential & Logarithm

Single-variable calculus from first principles

Two functions run the entire show in machine learning: the exponential eˣ and its inverse, the natural logarithm ln(x). They turn up in probabilities, in loss functions, in growth and decay. Getting comfortable with them now pays off everywhere later.

The defining feature of eˣ is that its rate of growth equals its current value — the bigger it is, the faster it climbs. That's what "exponential growth" really means: not just "fast," but growing in proportion to itself. The special number e ≈ 2.718 is the base for which this is exactly true.

The logarithm ln(x) simply undoes eˣ: it answers "e to what power gives me x?" So ln(eˣ) = x and e^{ln x} = x. Because they're inverses, their graphs are mirror images across the line y = x — drag the point in the figure and watch its reflection trace the other curve.

Where this lives in MLCross-entropy loss, the workhorse of classification, is built from −ln(p) where p is the probability the model assigned to the correct class. The log is there precisely because of the product-to-sum rule: the probability of a whole dataset is a giant product, and taking ln turns it into a sum the optimizer can differentiate term by term. "Log-likelihood" is exactly this trick.
▶ Exponential & Logarithm
← Lines & PolynomialsTrigonometric Functions →