Sequences

Single-variable calculus from first principles

A sequence is an ordered, never-ending list of numbers: a first term, a second, a third, and so on forever. We write the term in position n as aₙ, so the whole list is a₁, a₂, a₃, … The little subscript n is just a position label — term number 1, term number 2, term number 7.

Usually a sequence comes with a rule that tells you the term in any position. Plug a position in, get a number out. That makes a sequence really a function whose inputs are the counting numbers 1, 2, 3, …

Think of a savings account where the balance is recorded once a month: the first reading, the second, the third, and on forever. That ordered list of monthly balances is exactly a sequence, with aₙ meaning the balance in month n. If every month your balance creeps a little closer to a target you're saving toward, the sequence is settling onto that target — its limit.

Where this lives in MLA neural network's weights during training form a sequence: w₀, w₁, w₂, … — one entry per optimization step. When people say training "converges," they mean this sequence of weights is settling onto a limit. When they say it "diverges" (often because the learning rate was too high), they mean the blow-up picture: the terms grow instead of settling.
▶ Sequences
← Reading a GraphPartial Sums →