Independence

Conditional probability $P(A\mid B)$ measures how the occurrence of $B$ changes the probability of $A$. If that probability is unchanged — $P(A\mid B) = P(A)$ — we say $A$ is independent of $B$. Substituting the definition $P(A\mid B) = \dfrac{P(A\cap B)}{P(B)}$ and rearranging gives the symmetric form.

Definition 11.3. Events $A$ and $B$ are independent if $P(A\cap B) = P(A)\,P(B).$

This version is symmetric in $A$ and $B$ and needs no assumption that $P(B) > 0$. When $P(A) > 0$ it also follows that $P(B\mid A) = P(B)$ — the knowledge runs both ways. Independence is only meaningful for events in the same sample space; it makes no sense to ask whether events from unrelated experiments are independent.

These two ideas are often confused, but they are almost opposites. Mutually exclusive (disjoint) events cannot both happen: $A\cap B = \varnothing$, so $P(A\cap B) = 0$. Independent events satisfy $P(A\cap B) = P(A)\,P(B)$.

If $A$ and $B$ are disjoint and both have positive probability, then $P(A\cap B) = 0$ but $P(A)\,P(B) > 0$, so they are not independent. In fact, for disjoint events knowing that $A$ occurred tells you $B$ definitely did not — that is maximal dependence, not independence.

Proposition 11.3. If $A$ and $B$ are independent, then so are $A^c$ and $B$, $A$ and $B^c$, and $A^c$ and $B^c$.

The idea: $A\cap B$ and $A^c\cap B$ are disjoint and together make up $B$, so $P(A^c\cap B) = P(B) - P(A\cap B) = P(B) - P(A)P(B) = (1-P(A))P(B) = P(A^c)\,P(B)$. This is what lets us use the complement trick for independent events: the probability that at least one of several independent events occurs is $P\!\left(\bigcup_i A_i\right) = 1 - \prod_i \bigl(1 - P(A_i)\bigr),$ because the non-occurrences are independent too.

For more than two events, independence is subtler. Events $A_1,\dots,A_n$ are mutually independent if every sub-collection factorizes: for all $2 \le k \le n$, $P(A_{i_1}\cap\cdots\cap A_{i_k}) = P(A_{i_1})\cdots P(A_{i_k})$. For three events $A,B,C$ this means all four conditions hold: the three pairwise products and $P(A\cap B\cap C) = P(A)P(B)P(C)$.

They are pairwise independent if only every pair is independent. Pairwise independence is strictly weaker: it does not imply mutual independence (Example 11.8 below).