Conditional Probability & the Law of Total Probability

Suppose we learn that an event $B$ has occurred. Outcomes outside $B$ are now impossible, so we set their probability to $0$; the outcomes inside $B$ keep their relative likelihoods but must be rescaled so the total is again $1$. Dividing every original probability by $\mathbb{P}(B)$ does exactly this. Example 11.1 makes it concrete: with $\Omega=\{1,2,3\}$ and $\mathbb{P}\{1\}=\tfrac15,\ \mathbb{P}\{2\}=\tfrac25,\ \mathbb{P}\{3\}=\tfrac25$, learning $B=\{1,2\}$ gives $\mathbb{P}(B)=\tfrac15+\tfrac25=\tfrac35$, and $\mathbb{P}(1\mid B)=\frac{1/5}{3/5}=\frac13,\qquad \mathbb{P}(2\mid B)=\frac{2/5}{3/5}=\frac23.$ Outcome $2$ is still twice as likely as outcome $1$, exactly as before — only the scale changed. This is Definition 11.1: for any event $A$ and any $B$ with $\mathbb{P}(B)>0$, $\mathbb{P}(A\mid B)=\frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)}.$ The requirement $\mathbb{P}(B)>0$ is essential — conditioning on an impossible event is undefined. The map $\mathbb{P}(\cdot\mid B)$ is itself a genuine probability measure on $\Omega$, now concentrated on $B$: it satisfies all three axioms of probability.

When $\Omega$ has finitely many equally likely outcomes and $B\neq\varnothing$, both probabilities in the definition are counts divided by the same total $\#\Omega$, which cancels. Remark 11.1 records the shortcut: $\mathbb{P}(A\mid B)=\frac{\#(A\cap B)}{\#B}.$ Geometrically (Figure 11.1), conditioning on $B$ zooms in on $B$ and measures what fraction of it also lies in $A$ — the overlap $A\cap B$ as a share of $B$. Example 11.2 is a clean illustration: an urn has $4$ red and $6$ green balls and we draw $3$ without replacement. Let $A=\{\text{exactly }2\text{ red}\}$ and $B=\{\text{at least }1\text{ red}\}$. Since two red already means at least one, $A\subseteq B$, so $A\cap B=A$. Counting, $\#A=\binom{4}{2}\binom{6}{1}=36$ out of $\binom{10}{3}=120$, while $\#B=120-\binom{6}{3}=120-20=100$. Hence $\mathbb{P}(A\mid B)=\frac{36}{100}=\frac{9}{25}=0.36.$ The complement rule did the heavy lifting for $B$: it is far easier to count the outcomes with no red and subtract.

Clearing the denominator in Definition 11.1 turns a conditional probability into a product — the multiplication rule: $\mathbb{P}(A\cap B)=\mathbb{P}(A\mid B)\,\mathbb{P}(B)=\mathbb{P}(B\mid A)\,\mathbb{P}(A).$ It is tailor-made for experiments that unfold in stages, where each factor is the chance of the next step given everything so far. Telescoping the definition extends it to any number of events (the chain rule): $\mathbb{P}(A\cap B\cap C)=\mathbb{P}(A)\,\mathbb{P}(B\mid A)\,\mathbb{P}(C\mid A\cap B),$ provided no denominator vanishes. Example 11.3 draws balls without replacement from an urn of $8$ red and $4$ white. With $R_i=\{\text{draw }i\text{ is red}\}$, $\mathbb{P}(R_1\cap R_2)=\mathbb{P}(R_1)\,\mathbb{P}(R_2\mid R_1)=\frac{8}{12}\cdot\frac{7}{11}=\frac{14}{33}.$ The factor $\mathbb{P}(R_2\mid R_1)=\tfrac{7}{11}$ encodes that one red is already gone: $7$ reds remain among $11$ balls. The same bookkeeping gives longer runs, e.g. $\mathbb{P}(R_1R_2W_3W_4)=\tfrac{8}{12}\cdot\tfrac{7}{11}\cdot\tfrac{4}{10}\cdot\tfrac{3}{9}=\tfrac{28}{495}$.

Often $A$ is awkward to handle directly but easy given which of several scenarios occurred. A partition $\{B_1,\dots,B_n\}$ of $\Omega$ (Definition 11.2) is a family of pairwise disjoint events, $B_i\cap B_j=\varnothing$ for $i\neq j$, whose union is all of $\Omega$. It slices $A$ into disjoint pieces $A\cap B_i$, and additivity together with the multiplication rule gives the Law of Total Probability (Proposition 11.1): for a partition with every $\mathbb{P}(B_i)>0$, $\mathbb{P}(A)=\sum_{i=1}^{n}\mathbb{P}(A\cap B_i)=\sum_{i=1}^{n}\mathbb{P}(A\mid B_i)\,\mathbb{P}(B_i).$ The simplest partition is $\{B,B^c\}$ when $0<\mathbb{P}(B)<1$, giving $\mathbb{P}(A)=\mathbb{P}(A\mid B)\mathbb{P}(B)+\mathbb{P}(A\mid B^c)\mathbb{P}(B^c)$. Example 11.4 runs a two-stage experiment: choose Urn I ($2$ green, $1$ red) or Urn II ($2$ red, $3$ yellow), each with probability $\tfrac12$, then draw one ball. Conditioning on the urn, $\mathbb{P}(\text{red})=\tfrac13\cdot\tfrac12+\tfrac25\cdot\tfrac12=\tfrac16+\tfrac15=\tfrac{11}{30},$ and since only Urn I has green balls, $\mathbb{P}(\text{green})=\tfrac23\cdot\tfrac12+0=\tfrac13$. Note Definition 11.2 asks only for disjointness and full coverage; the positivity $\mathbb{P}(B_i)>0$ is an extra hypothesis that Proposition 11.1 adds so each conditional probability is defined.