Bayes' Formula: Reversing the Order of Conditioning

Conditional probability (Definition 11.1) gives $P(A\mid B)$: the chance of an effect $A$ once the cause $B$ is known. Frequently we observe the effect and want the cause, $P(B\mid A)$. Example 11.5 makes this concrete. Urn I holds $2$ green and $1$ red ball; urn II holds $2$ red and $3$ yellow balls. We pick an urn at random and draw a ball; given that it is red, which urn did it come from? The forward probabilities $P(\text{red}\mid\text{urn I})=\tfrac13$ and $P(\text{red}\mid\text{urn II})=\tfrac25$ are immediate, but the reverse $P(\text{urn I}\mid\text{red})$ is what we actually want. Rewrite it as $P(\text{urn I}\mid\text{red})=\dfrac{P(\text{red}\cap\text{urn I})}{P(\text{red})}=\dfrac{P(\text{red}\mid\text{urn I})P(\text{urn I})}{P(\text{red})}$ and expand $P(\text{red})$ with the law of total probability. The answer $\tfrac{5}{11}<\tfrac12$ says the red ball more likely came from urn II -- precisely because urn II is richer in red balls. Bayes' formula is the machine that performs this reversal in general.

Proposition 11.2 packages the reversal. Whenever $P(A),P(B),P(B^c)>0$, $P(B\mid A)=\frac{P(A\mid B)\,P(B)}{P(A\mid B)\,P(B)+P(A\mid B^c)\,P(B^c)}.$ The derivation is two moves. First, the definition of conditional probability (Definition 11.1) used twice: $P(B\mid A)=\dfrac{P(A\cap B)}{P(A)}$ and $P(A\cap B)=P(A\mid B)P(B)$, so the numerator is $P(A\mid B)P(B)$. Second, the law of total probability (Proposition 11.1) splits the denominator across the partition $\{B,B^c\}$: $P(A)=P(A\mid B)P(B)+P(A\mid B^c)P(B^c)$. So the scary-looking denominator is just $P(A)$ written in computable pieces. Everything required -- the two likelihoods $P(A\mid B),P(A\mid B^c)$ and the prior $P(B)$ -- is exactly the data a forward model already supplies.

Remark 11.2 names the two ends of the computation. The prior probability $P(B)$ is our belief before seeing evidence; the posterior probability $P(B\mid A)$ is that belief updated after observing $A$. Bayes' formula is the engine that turns one into the other. Whether evidence raises or lowers belief depends on how diagnostic $A$ is: if $A$ is more likely under $B$ than under $B^c$ (that is, $P(A\mid B)>P(A\mid B^c)$) the posterior exceeds the prior; if it is less likely, the posterior drops; and in the knife-edge case $P(A\mid B)=P(A\mid B^c)$ the evidence is irrelevant and the posterior equals the prior -- which is exactly independence of $A$ and $B$. The size of the shift, however, is still anchored by the prior, and that is the heart of the next idea.

The most famous use of Bayes is screening for a rare condition (Example 11.6). A test is described by its sensitivity $P(+\mid D)$ (true-positive rate), its false-positive rate $P(+\mid D^c)$ (so its specificity is $1-P(+\mid D^c)$), and the disease's prevalence $P(D)$, which plays the role of the prior. Intuition whispers that a $96\%$-sensitive test coming back positive means about a $96\%$ chance of disease. Bayes says otherwise. When the disease is rare, the enormous healthy population generates many false positives that swamp the few true positives, so the posterior $P(D\mid +)$ can sit far below $50\%$. Reading the sensitivity as the answer -- ignoring the prior $P(D)$ -- is the base-rate fallacy. The fix never changes: put the prevalence in the numerator and inside the total-probability denominator, and let the arithmetic correct the intuition.