Skip to content
VidiMaster it, module by module
Module 10/Multivariate expectation & variance

Linearity of expectation & variance of sums

Expectation of a sum is always the sum of expectations — no independence needed. Variance adds too, but only when the variables are uncorrelated. These give slick proofs of the binomial's mean and variance.

Before you start — give these a try

Attempting first primes your brain for the lesson — even if you miss. Nothing is graded or saved; it's just a warm-up.

X∼Bin(50,0.2)X\sim\mathrm{Bin}(50,0.2). Find Var(X)=np(1−p)\mathrm{Var}(X)=np(1-p).

X,YX,Y independent with Var(X)=4, Var(Y)=9\mathrm{Var}(X)=4,\ \mathrm{Var}(Y)=9. Find Var(X+Y)\mathrm{Var}(X+Y).

What you’ll be able to do

  • Apply linearity of expectation (Proposition 15.8), including the indicator trick.
  • Use Var(∑Xi)=∑Var(Xi)\mathrm{Var}(\sum X_i)=\sum\mathrm{Var}(X_i) for independent/uncorrelated variables.
  • Compute EE and Var\mathrm{Var} of the sample mean (Var(Xˉn)=σ2/n\mathrm{Var}(\bar X_n)=\sigma^2/n).

In your course

· MATH2015 · Linear Algebra & Probability
§15.4 Expectation and Variance in the Multivariate Setting
  • Proposition 15.8Linearity of expectation
    E[∑igi(Xi)]=∑iE[gi(Xi)]E[\sum_i g_i(X_i)]=\sum_i E[g_i(X_i)] (no independence needed).
  • Proposition 15.9Expectation of a product under independence
  • Proposition 15.10Variance of a sum of independent variables
  • Proposition 15.11Sample mean: E[Xˉn]=μE[\bar X_n]=\mu, Var(Xˉn)=σ2/n\mathrm{Var}(\bar X_n)=\sigma^2/n
  • Examples 15.7-15.9Mean & variance of the binomial via indicators
1

Linearity of expectation

For any random variables on the same space (Proposition 15.8),\n\nE[X1+⋯+Xn]=E[X1]+⋯+E[Xn],E[X_1+\cdots+X_n]=E[X_1]+\cdots+E[X_n],\n\nwith no independence assumption. The indicator trick writes a count as a sum of 0/10/1 indicators and adds their expectations.

2

Products and variance under independence

Independence lets expectation of a product factor, E[∏gk(Xk)]=∏E[gk(Xk)]E[\prod g_k(X_k)]=\prod E[g_k(X_k)] (Proposition 15.9). A consequence (Proposition 15.10): for independent (more generally, uncorrelated) variables,\n\nVar(X1+⋯+Xn)=Var(X1)+⋯+Var(Xn).\mathrm{Var}(X_1+\cdots+X_n)=\mathrm{Var}(X_1)+\cdots+\mathrm{Var}(X_n).

3

Mean & variance of the binomial

Write X∼Bin(n,p)X\sim\mathrm{Bin}(n,p) as a sum of nn independent Bernoulli(p)(p) indicators. Linearity gives E[X]=npE[X]=np; independence gives Var(X)=n p(1−p)\mathrm{Var}(X)=n\,p(1-p).

4

The sample mean

For i.i.d. X1,…,XnX_1,\dots,X_n with mean μ\mu, variance σ2\sigma^2, the sample mean Xˉn=1n∑Xi\bar X_n=\tfrac1n\sum X_i satisfies (Proposition 15.11)\n\nE[Xˉn]=μ,Var(Xˉn)=σ2n.E[\bar X_n]=\mu,\qquad \mathrm{Var}(\bar X_n)=\frac{\sigma^2}{n}.\n\nAveraging keeps the mean but shrinks the variance — the engine behind the law of large numbers.

Proposition 15.8 — Linearity of expectation

E[g1(X1)+⋯+gn(Xn)]=E[g1(X1)]+⋯+E[gn(Xn)]E[g_1(X_1)+\cdots+g_n(X_n)]=E[g_1(X_1)]+\cdots+E[g_n(X_n)]; in particular E[∑Xi]=∑E[Xi]E[\sum X_i]=\sum E[X_i]. No independence required.

Intuition. Expectation is a sum/integral, which is linear regardless of dependence.
Proposition 15.10 — Variance of a sum of independent variables

If X1,…,XnX_1,\dots,X_n are independent (finite variances), then Var(∑Xi)=∑Var(Xi)\mathrm{Var}(\sum X_i)=\sum \mathrm{Var}(X_i).

Intuition. Cross terms Cov(Xi,Xj)\mathrm{Cov}(X_i,X_j) vanish when the variables don't co-vary.
Proposition 15.11 — Sample mean

For i.i.d. XiX_i (mean μ\mu, variance σ2\sigma^2): E[Xˉn]=μE[\bar X_n]=\mu and Var(Xˉn)=σ2/n\mathrm{Var}(\bar X_n)=\sigma^2/n.

Intuition. Unbiased, and increasingly precise as nn grows.

Worked examples

Example 1

X∼Bin(n,p)X\sim\mathrm{Bin}(n,p). Use linearity to find E[X]E[X].

  1. 1

    Write X=X1+⋯+XnX=X_1+\cdots+X_n with Xi∼Bernoulli(p)X_i\sim\mathrm{Bernoulli}(p), so E[Xi]=pE[X_i]=p.

  2. 2

    By linearity E[X]=∑E[Xi]=npE[X]=\sum E[X_i]=np.

Answer. E[X]=npE[X]=np.
Example 2

Deal 55 cards without replacement; let XX be the number of aces. Find E[X]E[X].