Bridge to Integration

Single-variable calculus from first principles

In the last two lessons you added up a list of numbers and asked where the running total was heading. Now we make one bold leap: what if the things we're adding are infinitely many, infinitely thin pieces? That single move — sum tiny pieces, then take a limit — is the whole idea of the integral.

Here's the picture. You want the area under a curve, but the top is wavy, so there's no single height to multiply by the width. So you cheat, carefully: cover the region with thin vertical rectangles, each so narrow the curve is almost flat across it. Add up their areas. You won't get the exact answer — the rectangle tops poke above or fall below the curve — but you'll get close. Then make the rectangles thinner.

To find the area of an oddly-shaped region, imagine filling it with many thin vertical strips, like stacking a row of coins side by side under the curve. Each strip is so narrow that its top is almost flat, so you can treat it as a simple rectangle and add up the areas. The thinner you slice the strips — the smaller you make Δx — the more snugly the stack fills the region, and the area you get closes in on the exact answer.

Where this lives in MLThis is the bridge to all of continuous probability. An expectation E[f(X)] = ∫ f(x)p(x) dx is exactly this limit-of-a-sum, and when a model can't compute it exactly it falls back on Monte Carlo: replace the integral with an average over random samples, which is a Riemann-style sum. Every "average over a distribution" inside a generative model is approximating the picture above.
▶ Bridge to Integration
← Partial SumsLines & Polynomials →