Confidence Intervals

Inference, estimation, and decision-making from data

A point estimate like x̄ = 5.2 is almost certainly not exactly the true mean, so a single number alone is dishonest. A confidence interval reports a range plus a confidence level: "the true θ lies in [L, U], with 95% confidence." It quantifies how much your finite sample lets you trust the estimate.

The most common case uses the Central Limit Theorem: the sample mean is approximately normal, so the interval is the estimate plus-or-minus a margin of error:

The standard error σ/√n shrinks as n grows: four times the data halves the margin. The z-value sets the confidence: 1.96 for 95%, 2.576 for 99%.

Where this lives in MLThis is how honest ML papers report results. An accuracy of "91.2% ± 0.4%" is a confidence interval; the ± is the error bar. When two models' intervals overlap heavily, the "winner" may just be lucky sampling. When the population σ is unknown or the distribution is weird, the bootstrap (resampling your test set with replacement) builds the interval empirically, no formula needed.

▶ Confidence Intervals

← Bayesian Estimation Framework →