The Poisson & Exponential Distributions

The Poisson distribution models the number of times a rare event happens in a fixed stretch of time or space, when those events occur independently and at a constant average rate $\lambda>0$. By Definition 13.6, $X\sim\text{Poisson}(\lambda)$ takes values in $\{0,1,2,\dots\}$ with probability mass function $\mathbb{P}\{X=k\}=\frac{e^{-\lambda}\lambda^{k}}{k!},\qquad k=0,1,2,\dots.$ The parameter $\lambda$ is the expected count over the window, so it sets the whole shape at once. This really is a p.m.f.: every mass is positive, and the values sum to $1$ because the exponential series gives $\sum_{k=0}^{\infty}\frac{\lambda^{k}}{k!}=e^{\lambda}$, hence $\sum_{k=0}^{\infty}\frac{e^{-\lambda}\lambda^{k}}{k!}=e^{-\lambda}e^{\lambda}=1$. Classic settings are phone calls arriving at a call centre, radioactive decays in a second, or requests hitting a web server — any tally of independent events with no natural upper bound, each individually unlikely but with many opportunities to occur.

A striking feature of the Poisson law is that its mean and variance coincide: Proposition 13.8 states $\mathbb{E}[X]=\lambda$ and $\mathrm{Var}(X)=\lambda$. The mean follows by pulling one factor out and re-indexing: $\mathbb{E}[X]=\sum_{k=0}^{\infty}k\,\frac{e^{-\lambda}\lambda^{k}}{k!}=\lambda\sum_{k=1}^{\infty}\frac{e^{-\lambda}\lambda^{k-1}}{(k-1)!}=\lambda\sum_{j=0}^{\infty}\frac{e^{-\lambda}\lambda^{j}}{j!}=\lambda.$ The variance is cleanest through the factorial moment $\mathbb{E}[X(X-1)]$, where the same trick drops two factors: $\mathbb{E}[X(X-1)]=\sum_{k=2}^{\infty}\frac{e^{-\lambda}\lambda^{k}}{(k-2)!}=\lambda^{2}.$ Then $\mathbb{E}[X^{2}]=\mathbb{E}[X(X-1)]+\mathbb{E}[X]=\lambda^{2}+\lambda$, so $\mathrm{Var}(X)=\mathbb{E}[X^{2}]-(\mathbb{E}[X])^{2}=\lambda^{2}+\lambda-\lambda^{2}=\lambda.$ Equality of mean and variance is a useful fingerprint: if observed counts have a sample variance far from their sample mean, a Poisson model is suspect.

Where the Poisson counts events, the exponential distribution times them: it is the continuous analogue of the geometric distribution and models the waiting time until the next event — the time until the next customer arrives, or until a particle decays. By Definition 13.7, $X\sim\text{Exp}(\lambda)$ has density $f(x)=\begin{cases}\lambda e^{-\lambda x}, & x\ge 0,\\ 0, & x<0,\end{cases}$ with rate $\lambda>0$. Integrating the density gives the cumulative distribution function $F(t)=\mathbb{P}\{X\le t\}=1-e^{-\lambda t}$ for $t\ge 0$, and it is usually easiest to work from the tail $\mathbb{P}\{X>t\}=1-F(t)=e^{-\lambda t}.$ Evaluating the first two moments, $\mathbb{E}[X]=\int_{0}^{\infty}x\lambda e^{-\lambda x}\,dx=\frac{1}{\lambda}$ and $\mathbb{E}[X^{2}]=\frac{2}{\lambda^{2}}$, so the mean is $1/\lambda$ and the variance is $\mathrm{Var}(X)=\frac{2}{\lambda^{2}}-\frac{1}{\lambda^{2}}=\frac{1}{\lambda^{2}}$. A larger rate $\lambda$ means events arrive faster, shortening the expected wait $1/\lambda$. Most questions reduce to the tail: $\mathbb{P}\{a<X<b\}=e^{-\lambda a}-e^{-\lambda b}$, and the median solves $1-e^{-\lambda m}=\tfrac12$.

The exponential's defining peculiarity is that it is memoryless: how long you have already waited tells you nothing about how much longer you must wait. Proposition 13.9 states that for $X\sim\text{Exp}(\lambda)$ and any $s,t>0$, $\mathbb{P}\{X>t+s\mid X>t\}=\mathbb{P}\{X>s\}.$ The proof is a one-line conditional-probability calculation using the tail. Since $\{X>t+s\}\subseteq\{X>t\}$, their intersection is just $\{X>t+s\}$, so $\mathbb{P}\{X>t+s\mid X>t\}=\frac{\mathbb{P}\{X>t+s\}}{\mathbb{P}\{X>t\}}=\frac{e^{-\lambda(t+s)}}{e^{-\lambda t}}=e^{-\lambda s}=\mathbb{P}\{X>s\}.$ In words, the distribution of the remaining wait is identical to that of a fresh wait, regardless of elapsed time — a used component is as good as new. By Remark 13.2 the exponential is the only continuous distribution on $[0,\infty)$ with this property; the geometric distribution is its discrete counterpart, which is likewise memoryless.