Variance & Standard Deviation

The mean $\mu=\mathbb{E}[X]$ says where a random variable sits; it says nothing about how tightly its values cluster there. Variance fills that gap. Measuring a typical deviation $X-\mu$ directly is useless because departures above and below the mean cancel: $\mathbb{E}[X-\mu]=\mu-\mu=0$ for every $X$. The remedy is to square the deviation before averaging, so that every discrepancy counts positively. Definition 13.4 therefore sets $\mathrm{Var}(X)=\mathbb{E}\big[(X-\mu)^2\big],$ the mean squared deviation from $\mu$, written $\sigma^2$. For a discrete $X$, Proposition 13.2 turns this expectation into a weighted sum over the possible values, $\mathrm{Var}(X)=\sum_k (k-\mu)^2\,\mathbb{P}(X=k)$ (with the integral $\int_{-\infty}^{\infty}(x-\mu)^2 f(x)\,dx$ when $X$ has density $f$). Each term weighs how far a value lies from the mean, squared, by how likely that value is: a variable that can land far from $\mu$ with appreciable probability has a large variance, and one glued near $\mu$ a small one. Because the deviation is squared, variance carries the square of $X$'s units — one reason the standard deviation is often reported in its place.

Summing $(k-\mu)^2\mathbb{P}(X=k)$ works, but it is clumsy: it needs $\mu$ first and then a fresh pass squaring each centred value. Expanding the square gives a shortcut. Since $(X-\mu)^2=X^2-2\mu X+\mu^2$ and $\mu$ is a constant, linearity of expectation yields $\mathrm{Var}(X)=\mathbb{E}[X^2]-2\mu\,\mathbb{E}[X]+\mu^2=\mathbb{E}[X^2]-2\mu^2+\mu^2,$ which collapses to the computational formula of Proposition 13.3: $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mu^2=\mathbb{E}[X^2]-(\mathbb{E}[X])^2.$ In words, variance is the mean of the square minus the square of the mean. In practice you read two ordinary expectations off the p.m.f. — $\mu=\mathbb{E}[X]=\sum_k k\,\mathbb{P}(X=k)$ and the second moment $\mathbb{E}[X^2]=\sum_k k^2\,\mathbb{P}(X=k)$ — and subtract $\mu^2$. This is almost always the quicker computation by hand, and it is the route used throughout this lesson. It even carries a free sanity check: because $\mathrm{Var}(X)\ge 0$, you must always find $\mathbb{E}[X^2]\ge(\mathbb{E}[X])^2$, so a negative answer signals an arithmetic slip.

Variance is measured in the square of $X$'s units — square dollars, square seconds — which makes its size awkward to interpret next to the mean. Taking the square root cures this. The standard deviation is $\sigma=\mathrm{SD}(X)=\sqrt{\mathrm{Var}(X)},$ a nonnegative number back in the original units of $X$, representing a typical distance of $X$ from its mean. A small $\sigma$ means the distribution is concentrated near $\mu$; a large $\sigma$ means it is widely spread. Variance and standard deviation carry exactly the same information — each determines the other — but $\sigma$ is the one you can sensibly compare to $\mu$, or mark off on the same axis as the values of $X$. In practice you finish most variance computations by taking a square root to report $\sigma$.

Variance responds to the two basic transformations of a random variable in a memorable way. Proposition 13.4 states that for constants $a,b$, $\mathrm{Var}(aX+b)=a^2\,\mathrm{Var}(X).$ Two things stand out. First, the additive constant $b$ disappears: adding $b$ slides the whole distribution along the line without changing how spread out it is, so $\mathrm{Var}(X+b)=\mathrm{Var}(X)$ — spread is about distances between values, and a rigid shift preserves those distances. Second, the multiplier $a$ comes out squared, not linearly: stretching values by $a$ stretches every deviation by $a$, hence every squared deviation by $a^2$. At the level of the standard deviation this reads cleanly as $\mathrm{SD}(aX+b)=|a|\,\mathrm{SD}(X)$ (the absolute value because a standard deviation is never negative). A useful consequence is that $\mathrm{Var}(2X)=4\,\mathrm{Var}(X)$, not $2\,\mathrm{Var}(X)$. At the opposite extreme, Proposition 13.5 pins down when there is no spread at all: since $\mathrm{Var}(X)=\mathbb{E}[(X-\mu)^2]$ is the expectation of a nonnegative quantity it is always $\ge 0$, and it equals $0$ exactly when $X$ is almost surely constant, i.e. $\mathbb{P}(X=\mu)=1$. (The variances of the standard families — $p(1-p)$ for a Bernoulli, $np(1-p)$ for a binomial, $\tfrac{(b-a)^2}{12}$ for a uniform, $\tfrac{1-p}{p^2}$ for a geometric — are gathered in a separate lesson on the moments of named distributions.)