Skip to content
VidiMaster it, module by module
Module 10/Joint distributions

Joint distributions & marginals

The joint p.m.f. gives the probability of every combination of values of several random variables at once; summing it over one variable recovers the marginal distribution of the other.

Before you start — give these a try

Attempting first primes your brain for the lesson — even if you miss. Nothing is graded or saved; it's just a warm-up.

Two fair dice X1,X2X_1,X_2. Find P(X1=2, X2=2)P(X_1=2,\,X_2=2).

To obtain the marginal p.m.f. of XX from a joint table of (X,Y)(X,Y) you:

What you’ll be able to do

  • State the joint probability mass function and the two conditions it satisfies.
  • Read joint and event probabilities from a joint p.m.f. table.
  • Recover a marginal p.m.f. by summing the joint over the other variables.

In your course

· MATH2015 · Linear Algebra & Probability
§15.1 Joint Distribution of Discrete Random Variables§15.2 (marginals)
  • Definition 15.1Joint probability mass function
  • Proposition 15.1Expectation of a function of several variables (multivariate LOTUS)
  • Proposition 15.2Marginal probability mass function
  • Example 15.1Three fair coin flips
1

Joint probability mass function

For discrete random variables X1,…,XnX_1,\dots,X_n, the joint p.m.f. is\n\np(k1,…,kn)=P(X1=k1,…,Xn=kn).p(k_1,\dots,k_n)=P(X_1=k_1,\dots,X_n=k_n).\n\nIt must satisfy p(k1,…,kn)≥0p(k_1,\dots,k_n)\ge 0 and ∑k1,…,knp(k1,…,kn)=1\sum_{k_1,\dots,k_n}p(k_1,\dots,k_n)=1. In the two-variable case a table lists P(X=i,Y=j)P(X=i,Y=j) in each cell.

2

Probabilities from the joint p.m.f.

Any event about the variables is a sum of joint-p.m.f. entries. More generally (Proposition 15.1), for g:Rn→Rg:\mathbb{R}^n\to\mathbb{R},\n\nE[g(X1,…,Xn)]=∑k1,…,kng(k1,…,kn) p(k1,…,kn).E[g(X_1,\dots,X_n)]=\sum_{k_1,\dots,k_n}g(k_1,\dots,k_n)\,p(k_1,\dots,k_n).

3

Marginal distributions

The marginal p.m.f. of one variable is obtained by summing the joint over all the others (Proposition 15.2):\n\npXj(k)=∑other kip(k1,…,kn).p_{X_j}(k)=\sum_{\text{other }k_i}p(k_1,\dots,k_n).\n\nIn a table, the marginal of XX is the row sums and the marginal of YY is the column sums.

Definition 15.1 — Joint p.m.f.

p(k1,…,kn)=P(X1=k1,…,Xn=kn)p(k_1,\dots,k_n)=P(X_1=k_1,\dots,X_n=k_n), with p≥0p\ge 0 and ∑p=1\sum p=1.

Intuition. One number for each combination of values; together they form a probability distribution over Rn\mathbb{R}^n.
Proposition 15.2 — Marginal p.m.f.

pXj(k)=∑ki, i≠jp(k1,…,kn)p_{X_j}(k)=\sum_{k_i,\,i\ne j}p(k_1,\dots,k_n) — sum the joint over every other variable.

Intuition. Collapsing the table along one axis leaves the distribution of the remaining variable.

Worked examples

Example 1

Given the joint table with P(X=0,Y=0)=0.1P(X{=}0,Y{=}0){=}0.1, P(X=0,Y=1)=0.2P(X{=}0,Y{=}1){=}0.2, P(X=1,Y=0)=0.3P(X{=}1,Y{=}0){=}0.3, P(X=1,Y=1)=0.4P(X{=}1,Y{=}1){=}0.4, find the marginal P(X=1)P(X=1).

  1. 1

    Sum the X=1X{=}1 row over all YY: P(X=1)=P(X=1,Y=0)+P(X=1,Y=1)P(X{=}1)=P(X{=}1,Y{=}0)+P(X{=}1,Y{=}1).

  2. 2

    =0.3+0.4=0.7=0.3+0.4=0.7.

Answer. P(X=1)=0.7P(X=1)=0.7.
Example 2

Two fair dice. Find P(X1=3, X2=5)P(X_1=3,\,X_2=5).