Skip to content
VidiMaster it, module by module
Module 10/Joint distributions

Independence of random variables

Random variables are independent exactly when their joint p.m.f. factors into the product of the marginals — for every combination of values, not just one.

Before you start — give these a try

Attempting first primes your brain for the lesson — even if you miss. Nothing is graded or saved; it's just a warm-up.

Consider two random variables X,Y∈{0,1}X,Y\in\{0,1\} with joint p.m.f.\n\n| | Y=0Y{=}0 | Y=1Y{=}1 |\n|---|---|---|\n| X=0X{=}0 | 0.10.1 | 0.20.2 |\n| X=1X{=}1 | 0.30.3 | 0.40.4 |\n\nCompute P(X=0) P(Y=0)P(X=0)\,P(Y=0) (the product of marginals for that cell).

Consider two random variables X,Y∈{0,1}X,Y\in\{0,1\} with joint p.m.f.\n\n| | Y=0Y{=}0 | Y=1Y{=}1 |\n|---|---|---|\n| X=0X{=}0 | 0.10.1 | 0.20.2 |\n| X=1X{=}1 | 0.30.3 | 0.40.4 |\n\nSince P(X=0,Y=0)=0.1P(X{=}0,Y{=}0)=0.1 but P(X=0)P(Y=0)=0.12P(X{=}0)P(Y{=}0)=0.12, XX and YY are independent.

What you’ll be able to do

  • State the factorization criterion for independence (Proposition 15.3).
  • Test independence from a joint table.
  • Distinguish independence of random variables from independence of particular events.

In your course

· MATH2015 · Linear Algebra & Probability
§15.2 Joint Distributions and Independence
  • Proposition 15.3Independence via the joint p.m.f. (factorization)
    X1,…,XnX_1,\dots,X_n independent   ⟺  p(k1,…,kn)=∏jpXj(kj)\iff p(k_1,\dots,k_n)=\prod_j p_{X_j}(k_j) for all values.
  • Proposition 15.4Independence of functions of disjoint variables
  • Remark 15.1A value of one variable need not pair with every value of another
  • Example 15.2Two dice: X1X_1 and S=X1+X2S=X_1+X_2 are dependent
1

Independence via factorization

Discrete X1,…,XnX_1,\dots,X_n are independent iff their joint p.m.f. factors into the marginals for all values (Proposition 15.3):\n\np(k1,…,kn)=pX1(k1)⋯pXn(kn)for every (k1,…,kn).p(k_1,\dots,k_n)=p_{X_1}(k_1)\cdots p_{X_n}(k_n)\quad\text{for every }(k_1,\dots,k_n).\n\nA single pair factoring is not enough — it must hold for every combination.

2

Random variables vs. events

Independence of the variables XX and YY is stronger than independence of one particular pair of events. Two events like {X1=1}\{X_1=1\} and {S=7}\{S=7\} can be independent even though the variables X1X_1 and S=X1+X2S=X_1+X_2 are not independent (because some other pair, e.g. {X1=1},{S=12}\{X_1=1\},\{S=12\}, fails to factor).

3

Functions of disjoint variables

If X1,…,Xm+nX_1,\dots,X_{m+n} are independent and Y=f(X1,…,Xm)Y=f(X_1,\dots,X_m), Z=g(Xm+1,…,Xm+n)Z=g(X_{m+1},\dots,X_{m+n}) use disjoint blocks, then YY and ZZ are independent (Proposition 15.4).

Proposition 15.3 — Independence via the joint p.m.f.

X1,…,XnX_1,\dots,X_n are independent iff p(k1,…,kn)=∏jpXj(kj)p(k_1,\dots,k_n)=\prod_{j} p_{X_j}(k_j) for every (k1,…,kn)(k_1,\dots,k_n).

Intuition. Knowing one variable tells you nothing about the others precisely when the joint is the product of marginals everywhere.
Proposition 15.4 — Functions of disjoint variables

If the XiX_i are independent, then ff of one block and gg of a disjoint block are independent random variables.

Intuition. Non-overlapping inputs carry no shared randomness, so their outputs stay independent.

Worked examples

Example 1

For the joint table (P(X=0,Y=0)=0.1P(X{=}0,Y{=}0){=}0.1, marginals P(X=0)=0.3P(X{=}0){=}0.3, P(Y=0)=0.4P(Y{=}0){=}0.4), are XX and YY independent?

  1. 1

    Independence needs P(X=0,Y=0)=P(X=0)P(Y=0)P(X{=}0,Y{=}0)=P(X{=}0)P(Y{=}0).

  2. 2

    P(X=0)P(Y=0)=0.3×0.4=0.12P(X{=}0)P(Y{=}0)=0.3\times 0.4=0.12, but the cell is 0.10.1.

  3. 3

    0.1≠0.120.1\ne 0.12, so they are not independent.

Answer. Not independent.
Example 2

Two fair dice X1,X2X_1,X_2 and S=X1+X2S=X_1+X_2. Are X1X_1 and SS independent?