Modules

Vectors in $\mathbb{R}^n$ are lists of numbers you can add and scale component by component. This lesson builds the algebra and geometry of vector addition, scalar multiplication, and linear combinations, then pins down the properties that make these operations so predictable.

Learn to describe lines and planes in three-dimensional space using points and direction vectors, move fluidly between parametric, vector, and normal forms, and test whether a given point lies on a line or plane.

A subspace is a subset of a vector space that is itself a vector space under the very same addition and scalar multiplication. You never recheck all the axioms — you run one quick three-part test: it must contain the zero vector, and be closed under addition and under scalar multiplication.

A linear combination is what you get by scaling vectors and adding them; the span collects every possible linear combination of a set of vectors. Span is always a subspace through the origin, and its geometric shape — point, line, plane, or all of space — reflects how many independent directions the vectors supply.

Deciding whether a vector $b$ lies in $\text{span}\{v_1,\ldots,v_k\}$ comes down to one question: is the linear system $[\,v_1\ \cdots\ v_k\mid b\,]$ consistent? This lesson shows how to set it up, read the answer from row reduction, and recover the coefficients when $b$ is in the span.

A set of vectors is linearly independent when none of them is "redundant" — no vector can be built from the others. This lesson defines independence precisely, shows how to test for it with the homogeneous system, rank, and determinant, and teaches you to extract an explicit dependency relation when one exists.

A basis is the smallest set of vectors that still describes an entire space: linearly independent (no redundancy) and spanning (nothing left out). This lesson shows how to recognize a basis, why every basis of $\mathbb{R}^n$ has exactly $n$ vectors, and how to test a given set fast.

The dimension of a vector space is the number of vectors in any basis — a single number that counts its independent directions, equals the rank of any spanning set, and is the same no matter which basis you pick.

A basis gives every vector a unique "address" — its coordinate vector. This lesson shows how to compute $[v]_B$ by solving $Bc = v$ and how to rebuild $v$ from its coordinates, all in $\mathbb{R}^2$ and $\mathbb{R}^3$.

A linear map is a function between vector spaces that preserves addition and scalar multiplication. This lesson defines it precisely, shows how to test a formula for linearity, and separates genuine linear maps from affine and nonlinear impostors.

The kernel of a linear map collects every input that gets sent to zero, and its size tells you instantly whether the map is injective. Learn to compute it by solving $Ax=0$ and to read off injectivity from its dimension.

The image of a linear map is the set of all outputs it can actually produce — exactly the span of its matrix's columns. Measuring that image with the rank tells you instantly whether the map is surjective and whether a particular vector can be reached.

The Rank–Nullity Theorem says that for a linear map out of a finite-dimensional space, the dimensions of its kernel and image always add up to the dimension of the domain. This one equation lets you compute any missing count and instantly rule out impossible injective or surjective maps.

Every linear map $T:\mathbb{R}^n\to\mathbb{R}^m$ is "secretly" multiplication by a single matrix $A$, built by seeing what $T$ does to the standard basis vectors. This lesson shows how to build that matrix, use it to apply the map, and why composing maps means multiplying matrices.

Every scaling, rotation, reflection, and shear of the plane is captured by a single $2\times 2$ matrix, and applying the transformation is just matrix-vector multiplication. This lesson teaches you to read a matrix as a geometric motion and to build the matrix for the motion you want.

Matrices can be added, scaled, and multiplied, but the rules are not the ones from ordinary arithmetic. This lesson covers entrywise addition and scalar multiplication, the dimension rule and row-by-column recipe for matrix multiplication, and why $AB$ and $BA$ usually differ.

The inverse $A^{-1}$ is the matrix that undoes $A$. This lesson shows you when a $2\times 2$ matrix has an inverse, how to compute it with a one-line formula, and how inverses behave under multiplication.

The transpose flips a matrix across its main diagonal, swapping every row with the corresponding column. This lesson shows how to compute it, the clean algebra it obeys (including the order-reversing product rule), and how it defines symmetric matrices ($A=A^T$) and skew-symmetric matrices ($A=-A^T$).

Rank counts the independent directions a matrix produces (its pivots), while nullity counts the directions it collapses. The Rank–Nullity Theorem ties them together: for an $m\times n$ matrix, $\operatorname{rank}(A)+\operatorname{nullity}(A)=n$.

Master the three elementary row operations, encode each one as an elementary matrix, and see Theorem 5.2 turn every row operation into a single left-multiplication $A \to EA$ — the engine behind Gaussian elimination.

The staircase shape at the heart of Gaussian elimination: what row-echelon form means, how the recursive Gauss algorithm produces it step by step, and how counting pivots gives you the rank of any matrix.

Row echelon form gives you a staircase of pivots, but reduced row echelon form (rref) goes further: every pivot is scaled to a leading 1 and stands alone in its column. This lesson covers Definition 5.4, the Gauss–Jordan sweep that produces the rref, and Remark 5.2 — the fact that the rref of a matrix is unique.

Once you can row-reduce a matrix, you can invert one. This lesson turns Gauss–Jordan elimination into a machine for computing $A^{-1}$: augment $A$ with the identity, reduce the left block to $I_n$, and read the inverse off the right block. You will also learn to spot, mid-reduction, exactly when a matrix has no inverse at all.

Write any linear system as $Ax=b$, decide whether it is consistent using the rank criterion $\operatorname{rank}(A)=\operatorname{rank}([A\,|\,b])$, and describe its whole solution set as a particular solution plus the kernel.

How to find every solution of a linear system $Ax=b$. We justify why row-reducing the augmented matrix $[A\mid b]$ never changes the solution set (Lemma 5.1 and Theorem 5.5), read off consistency by comparing $\operatorname{rank}(A)$ with $\operatorname{rank}([A\mid b])$, and run the course's 4-step procedure to write the full solution as a particular solution plus a basis of $\ker A$. Along the way we meet the three possible outcomes: a unique solution, infinitely many, or none.

The determinant condenses a square matrix into a single number that tells you whether the matrix is invertible and measures the area or volume its rows span. This lesson builds the determinant from the $2\times 2$ formula $ad-bc$, its geometric meaning, the three defining axioms (Definition 6.1), and the Rule of Sarrus for $3\times 3$ matrices.

Two reliable ways to compute the determinant of a square matrix: row-reduce to triangular form and multiply the pivots, or expand in cofactors along a convenient row or column. Includes the exact rules for how each elementary row operation changes the value, and why $\det(A)\neq 0$ means $A$ is invertible.

The determinant is more than a number you grind out by row reduction: it is a multiplicative invariant that detects invertibility and behaves predictably under products, powers, inverses, transposes, and change of basis. This lesson (§6.2) collects the key algebraic properties of $\det$ and turns them into fast computational tools.

An enrichment tour beyond the three axioms (D1)-(D3): the Leibniz (permutation) formula for the determinant, why block-triangular matrices simply multiply their determinants, and how Cramer's rule and the Vandermonde determinant turn determinants into problem-solving tools.

Meet the equation $A\mathbf{v}=\lambda\mathbf{v}$. This lesson explains what eigenvalues and eigenvectors are, why the eigenspace $E_\lambda$ is exactly the kernel of $A-\lambda I$, and how the characteristic polynomial $c_A(\lambda)=\det(A-\lambda I)$ turns the search for eigenvalues into solving a single polynomial equation.

Two eigenvalues can be the same number yet behave differently. This lesson separates algebraic multiplicity (how often $\lambda$ is a root of the characteristic polynomial) from geometric multiplicity (how many independent eigenvectors it has), uses the eigenvalues to read off the trace and determinant, pins down the bound $1\le\operatorname{gemu}(\lambda)\le\operatorname{almu}(\lambda)\le n$, and shows which spectral data survive transposing or passing to a similar matrix.

A square matrix is diagonalizable when it is similar to a diagonal matrix, $A=SDS^{-1}$. This lesson shows how an eigenbasis supplies the columns of $S$ and the diagonal of $D$, how to decide whether a matrix is diagonalizable by matching geometric and algebraic multiplicities, and why the factorization makes powers $A^k$ cheap.

An enrichment look at subspaces that a matrix maps into themselves. We define $A$-invariant subspaces, see why one-dimensional invariant subspaces are exactly the eigenlines, describe the invariant subspaces of a diagonalizable matrix, and read off the axis and perpendicular plane of a 3D rotation.

Use eigenvalues and diagonalization to solve discrete systems $x(t+1)=Ax(t)$ and continuous systems $\frac{dx}{dt}=Ax$ in closed form, and to read off their stability and long-run behavior directly from the eigenvalues.

A Markov chain moves between states by fixed probabilities, encoded in a column-stochastic transition matrix $A$ that updates the state via $x(k+1)=Ax(k)$. When $A$ is regular, the chain always settles to a unique equilibrium distribution $x^{*}$ (the normalized eigenvector for $\lambda=1$) -- the same idea behind Google's PageRank. The matrix-power trick also counts paths in a directed graph.

An inner product turns two vectors into a single scalar, and from it we build the norm $\|v\|=\sqrt{\langle v,v\rangle}$ — unlocking length, distance, the Cauchy–Schwarz inequality, orthogonality, the Pythagorean theorem, and the angle $\cos\theta=\dfrac{u\cdot v}{\|u\|\,\|v\|}$ between vectors.

Finding a vector's coordinates in a basis usually means solving a linear system — tedious and error-prone in high dimensions. This lesson shows why an orthonormal basis makes that work vanish: each coordinate is a single dot product, $c_i=\mathbf{v}\cdot\mathbf{u}_i$ (Theorem 8.5). Along the way we define orthogonal and orthonormal sets (Definition 8.7), see why orthonormal vectors are automatically linearly independent (Proposition 8.1) and form bases (Theorem 8.4), and learn to normalize any orthogonal basis (Remark 8.4).

Every vector $\mathbf{x}\in\mathbb{R}^n$ splits uniquely as $\mathbf{x}=\mathbf{x}_{\parallel}+\mathbf{x}_{\perp}$: a part $\mathbf{x}_{\parallel}=\operatorname{proj}_V\mathbf{x}$ lying in a subspace $V$ and a part $\mathbf{x}_{\perp}$ orthogonal to it. This lesson builds that decomposition from an orthonormal basis, turns it into the symmetric projection matrix $P=QQ^{T}=A(A^{T}A)^{-1}A^{T}$, measures the distance from $\mathbf{x}$ to $V$ as $\|\mathbf{x}-\operatorname{proj}_V\mathbf{x}\|$, and relates the orthogonal complement $V^{\perp}$ to $V$ through $\dim V+\dim V^{\perp}=n$ and $\operatorname{im}(A)^{\perp}=\ker(A^{T})$.

When $A\mathbf{x}=\mathbf{b}$ is inconsistent — $\mathbf{b}\notin\operatorname{im}(A)$ — there is no exact solution, so we settle for the next best thing: the vector $\hat{\mathbf{x}}$ that makes $A\hat{\mathbf{x}}$ as close to $\mathbf{b}$ as possible. This lesson shows that the closest point in a subspace is the orthogonal projection (Theorem 8.12), that the minimiser $\hat{\mathbf{x}}$ is found by solving the normal equation $A^{\mathsf{T}}A\hat{\mathbf{x}}=A^{\mathsf{T}}\mathbf{b}$ (Theorem 8.13), and how this turns messy, over-determined data into a single clean line of best fit.

Orthonormal bases are wonderful to compute with — but where do they come from? The Gram–Schmidt process manufactures one from any basis, normalizing $\mathbf{v}_1$ and then, for each later vector, subtracting its projection onto the span of the vectors already processed before normalizing what remains (Theorem 8.15). The result depends on the order of the inputs (Remark 8.6), and when the vectors are the columns of a matrix the whole computation repackages into the QR factorization $A=QR$, with $Q$ orthonormal and $R$ upper triangular with positive diagonal (Theorem 8.16).

Orthogonal maps are the linear transformations of $\mathbb{R}^n$ that preserve the inner product: $\langle f(\mathbf u),f(\mathbf v)\rangle=\langle \mathbf u,\mathbf v\rangle$. This lesson shows why that single condition is the same as preserving length (Theorem 8.17) and angle (Corollary 8.2), why it is equivalent to sending the standard basis to an orthonormal basis (Theorem 8.18), and how it becomes the matrix criterion $A^{T}A=I$, equivalently $A^{-1}=A^{T}$ with orthonormal columns (Theorems 8.19, 8.21). Rotations and reflections are the $2\times 2$ picture, and every orthogonal matrix has $\det=\pm 1$ (Corollary 8.3).

Symmetric matrices ($A=A^T$) are the best-behaved operators in all of linear algebra, and this lesson uncovers why. Two structural miracles set the stage for the Spectral Theorem: every eigenvalue of a symmetric matrix is real (Proposition 9.2), and eigenvectors belonging to distinct eigenvalues are automatically orthogonal (Lemma 9.1). Along the way we see that orthogonal diagonalizability forces symmetry (Remark 9.1), and that the orthogonal complement of an eigenvector is $A$-invariant (Lemma 9.2) — the geometric engine that will power the full theorem next time.

A real square matrix can be diagonalized by an orthogonal change of basis exactly when it is symmetric. The Spectral Theorem (Theorem 9.1) packages a symmetric $A$ as $A=QDQ^{T}=QDQ^{-1}$, where $D$ holds the (always real) eigenvalues and the columns of the orthogonal matrix $Q$ form an orthonormal eigenbasis. This lesson shows why symmetry is the exact dividing line, and how to build $Q$ and $D$ in practice.

Diagonalization is reserved for square, symmetric matrices — yet every matrix $A$ has singular values. The trick is to pass to the symmetric matrix $A^TA$, whose eigenvalues $\lambda_i$ are always real and nonnegative (Lemma 9.3). Their square roots $\sigma_i=\sqrt{\lambda_i}$ are the singular values (Definition 9.2); an orthonormal eigenbasis $\mathbf{v}_i$ of $A^TA$ sends the unit sphere to an ellipse with semi-axes $\sigma_i$, because $\|A\mathbf{v}_i\|=\sigma_i$ (Theorem 9.2); and the number of nonzero $\sigma_i$ is exactly the rank of $A$ (Proposition 9.3).

The Spectral Theorem factors a symmetric matrix as $A=SDS^T$, but it is confined to square — indeed symmetric — matrices. The singular value decomposition lifts that restriction: every matrix, square or rectangular, factors as $A=U\Sigma V^T$ with $U$ and $V$ orthogonal and $\Sigma$ a diagonal matrix of singular values $\sigma_1\ge\sigma_2\ge\cdots\ge 0$. This lesson builds the SVD from the eigenvectors of $A^TA$, rewrites it as a sum of rank-one pieces $A=\sum_i\sigma_i\,u_iv_i^T$, and reads off its geometry: $A$ sends the unit sphere to an ellipsoid whose semi-axes are the singular values.

PCA is this module's capstone: it takes a cloud of high-dimensional data and finds the few directions that matter. Center the features into a matrix $A$, form the covariance matrix $C=\frac{1}{n-1}AA^{\mathsf T}$, and — because $C$ is symmetric — use the Spectral Theorem (or the SVD) to diagonalize it, $C=SDS^{\mathsf T}$. The orthonormal eigenvectors are the principal components; their eigenvalues $\lambda_k$ are the variances captured along each one. Keeping the components with the largest eigenvalues compresses the data with minimal loss — turning a point cloud in $\mathbb R^{75}$ into a picture in $\mathbb R^2$.

Every probability question begins with a probability space $(\Omega,\mathcal{F},\mathbb{P})$ (Definition 10.1): a sample space $\Omega$ holding every possible outcome $\omega$, a collection of events $\mathcal{F}$ (the subsets of $\Omega$ we assign probabilities to), and a probability measure $\mathbb{P}$ obeying three axioms — every probability lies in $[0,1]$, the certain event has $\mathbb{P}(\Omega)=1$ (and the impossible event $\mathbb{P}(\varnothing)=0$), and the probabilities of disjoint events add. This lesson translates experiments into the language of sets, unpacks the axioms and their first consequences (Remark 10.1), and — when $\Omega$ is finite with equally likely outcomes — collapses probability into pure counting via $\mathbb{P}(A)=\tfrac{|A|}{|\Omega|}$ (Proposition 10.1). We put the machinery to work on a fair die (Example 10.1), a pair of distinguishable dice (Example 10.2), and an urn draw (Example 10.3).

Once the axioms are in place, almost every probability computation runs on a short list of rules, and this lesson assembles them. The complement rule $\mathbb{P}(A^c)=1-\mathbb{P}(A)$ turns a hard event into an easy one \u2014 the engine behind the complement trick $\mathbb{P}(\text{at least one})=1-\mathbb{P}(\text{none})$, used in Example 10.4 to find the chance a value repeats when a fair die is rolled four times. For mutually exclusive (disjoint) events the Disjoint Sum Rule adds probabilities outright; when events overlap, the addition rule (the General Sum Rule) corrects for the double-counted intersection, $\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B)$. Remark 10.3 rearranges this to recover $\mathbb{P}(A\cap B)$ and extends it to the inclusion\u2013exclusion principle for three events, which (for two) we apply in Example 10.5.

Probability on a finite, equally likely sample space collapses to counting: $P(A)=\#A/\#\Omega$. This lesson organizes that counting around two yes/no questions about how a sample of size $k$ is drawn from $n$ objects --- does order matter? and is each object replaced before the next draw? --- giving a $2\times2$ grid of four sampling schemes. Three of them produce equally likely outcomes and power every probability here: ordered-with-replacement ($n^k$, Example 10.6), ordered-without-replacement ($n!/(n-k)!$, Example 10.7), and unordered-without-replacement ($\binom{n}{k}$, Example 10.8). We count each scheme, see how dividing an ordered count by $k!$ produces combinations, and turn the counts into probabilities --- including Jordan's books solved two ways (Example 10.9, Remark 10.4) and the three contrasting scenarios of Example 10.10.

New information reshapes a probability model: once we learn that an event $B$ has occurred, outcomes outside $B$ become impossible and the survivors must be renormalized so their probabilities again sum to $1$. Definition 11.1 packages this as $\mathbb{P}(A\mid B)=\mathbb{P}(A\cap B)/\mathbb{P}(B)$, valid whenever $\mathbb{P}(B)>0$. Rearranging gives the multiplication rule $\mathbb{P}(A\cap B)=\mathbb{P}(A\mid B)\mathbb{P}(B)$ — the engine behind sequential draws without replacement — and splitting $\Omega$ into a partition (Definition 11.2) yields the Law of Total Probability (Proposition 11.1), $\mathbb{P}(A)=\sum_i\mathbb{P}(A\mid B_i)\mathbb{P}(B_i)$, the backbone of the two-stage, two-urn reasoning in Example 11.4.

Conditional probability answers $P(A\mid B)$; Bayes' formula (Proposition 11.2) answers the reverse question $P(B\mid A)$ -- the probability of a cause given an observed effect. Assembled from the definition of conditional probability (Definition 11.1) and the law of total probability (Proposition 11.1), it updates a prior belief $P(B)$ into a posterior $P(B\mid A)$ once evidence $A$ arrives (Remark 11.2). The centerpiece is the medical-test / base-rate problem (Example 11.6), where a highly accurate test for a rare disease returns a surprisingly low posterior -- the base-rate fallacy that routinely fools intuition.

Two events are independent when knowing one tells you nothing about the other: P(A∩B) = P(A)·P(B).

A random variable (Definition 12.1) is not a number but a function $X:\Omega\to\mathbb{R}$ that reads each outcome $\omega$ of an experiment and reports a real value $X(\omega)$ — the sum of two dice, a gambler's change in wealth, a count. Writing $\{X\in B\}=\{\omega\in\Omega:X(\omega)\in B\}$ turns a question about values into an event with a probability $\mathbb{P}\{X\in B\}$, and the whole family of these probabilities is the probability distribution of $X$ (Definition 12.2). This lesson concentrates on discrete random variables (Definition 12.3), whose values form a finite or countably infinite list that carries all the probability. For them the distribution collapses to a single object, the probability mass function $p(k)=\mathbb{P}\{X=k\}$ (Definition 12.4): every probability becomes a sum of masses, $\mathbb{P}\{X\in B\}=\sum_{k\in B}p(k)$, and a function is a legitimate p.m.f. exactly when it is nonnegative and sums to $1$. We build p.m.f.s and read probabilities off them for a pair of dice (Examples 12.1, 12.3), a wealth/payoff game (Examples 12.2, 12.4), and a biased spinner. (Continuous random variables and densities are left for a later lesson.)

A continuous random variable $X$ (Definition 12.5) draws its probabilities not from a list of point masses but from the area under a curve: there is a probability density function (pdf) $f:\mathbb{R}\to\mathbb{R}$ with $\mathbb{P}(X\le b)=\int_{-\infty}^{b}f(x)\,dx\qquad\text{for all }b\in\mathbb{R},$ and more generally $\mathbb{P}(X\in B)=\int_B f(x)\,dx$. This lesson pins down what makes a function a legitimate density (Remark 12.1: $f\ge 0$ everywhere and total area $\int_{-\infty}^{\infty}f(x)\,dx=1$), reads probabilities off as areas $\mathbb{P}(a\le X\le b)=\int_a^b f(x)\,dx$, and draws the defining contrast with the discrete world — for a continuous variable every single point is negligible, $\mathbb{P}(X=c)=0$ (Proposition 12.1), so endpoints never matter: $\mathbb{P}(a\le X\le b)=\mathbb{P}(a<X<b)$. We anchor everything in the uniform distribution on $[0,1]$ (Example 12.5), where $f(x)=1$ on $[0,1]$ and a probability is just a length, and in a worked exponential density (Example 12.6).

Unlike the probability mass function (discrete variables only) or the density (continuous variables only), the cumulative distribution function (c.d.f.) is defined for every random variable (Definition 12.6): $F(x)=\mathbb{P}\{X\le x\}$ records the total probability accumulated up to the point $x$ — the 'sum-or-area-so-far' function. The '$\le$' is deliberate (Remark 12.2): it includes the endpoint and makes the c.d.f. deliver interval probabilities, $\mathbb{P}\{a<X\le b\}=F(b)-F(a)$. For a discrete variable the c.d.f. is a step function (Remark 12.3) that is flat between the possible values and jumps by $\mathbb{P}\{X=x\}$ at each one, as for the change-in-wealth variable of Example 12.7; for a continuous variable it rises smoothly, obtained by integrating the density, as for the $\mathrm{Unif}[1,3]$ variable of Example 12.8. Whatever its type, every c.d.f. shares the same three shape properties (Proposition 12.2): it is nondecreasing, right-continuous, and climbs from $0$ at $-\infty$ to $1$ at $+\infty$.

Repeated independent trials with just two outcomes — success (probability $p$) and failure (probability $1-p$) — are the seed from which two fundamental discrete distributions grow. The Bernoulli distribution (Definition 12.7) records a single such trial: a random variable $X\in\{0,1\}$ with $\mathbb{P}(X=1)=p$ and $\mathbb{P}(X=0)=1-p$, written $X\sim\mathrm{Ber}(p)$. Run the trial $n$ independent times and you obtain $n$ independent Bernoulli variables $X_1,\dots,X_n$; their sum $S_n=X_1+\cdots+X_n$ counts the successes, and its distribution is the Binomial (Definition 12.8): $\mathbb{P}(X=k)=\binom{n}{k}p^k(1-p)^{n-k}$ for $k=0,1,\dots,n$, written $X\sim\mathrm{Bin}(n,p)$. The binomial coefficient $\binom{n}{k}$ counts the arrangements of $k$ successes among the $n$ trials, each arrangement carrying probability $p^k(1-p)^{n-k}$, and the binomial theorem guarantees these probabilities sum to $1$. We close with Example 12.9: the chance that five rolls of a fair die yield two or three sixes, with $S_5\sim\mathrm{Bin}(5,\tfrac16)$, equal to $\tfrac{1500}{7776}\approx 0.193$.

Independent repeated Bernoulli trials — each a success with probability $p$, a failure with probability $1-p$ — raise a question the binomial does not answer: not how many successes in a fixed number of trials, but how long until a success. The geometric distribution (Definition 12.9) answers the first version: $X\sim\mathrm{Geom}(p)$ is the trial on which the first success occurs, with probability mass function $\mathbb{P}\{X=k\}=(1-p)^{k-1}p$ for $k=1,2,3,\dots$ — the price of $k-1$ failures followed by one success. Its masses form a geometric series summing to $1$, and its tail has a clean closed form $\mathbb{P}\{X>k\}=(1-p)^k$ (all of the first $k$ trials fail), so $\mathbb{P}\{X\le k\}=1-(1-p)^k$; Example 12.10 uses this to find the chance of needing more than seven rolls of a fair die for the first six, $(5/6)^7\approx0.279$. The negative binomial distribution (Definition 12.10) generalises the idea to the trial of the $r$-th success: $X\sim\mathrm{NegBin}(r,p)$ has $\mathbb{P}\{X=k\}=\binom{k-1}{r-1}p^r(1-p)^{k-r}$ for $k=r,r+1,\dots$, the coefficient $\binom{k-1}{r-1}$ counting the arrangements of the $r-1$ earlier successes among the first $k-1$ trials. Remark 12.4 ties the two together: the geometric is exactly $\mathrm{NegBin}(1,p)$.

These are the first two continuous distributions of the course, and both compute probabilities as areas under a density curve rather than by summing point masses. The continuous uniform distribution $X\sim\mathrm{Unif}(a,b)$ (Definition 12.11) spreads probability evenly over an interval $[a,b]$: its density is the constant $\frac{1}{b-a}$, so the chance of landing in a subinterval is just its length ratio, $P\{c\le X\le d\}=\frac{d-c}{b-a}$. The standard normal (Gaussian) distribution $Z\sim N(0,1)$ (Definition 12.12) is the famous bell curve, with density $\varphi(x)=\frac{1}{\sqrt{2\pi}}e^{-x^2/2}$ symmetric about $0$. Its cumulative distribution function $\Phi$ has no closed form, so we read $\Phi$ from a standard-normal table or technology and assemble every probability from $P\{a\le Z\le b\}=\Phi(b)-\Phi(a)$ together with the symmetry rule $\Phi(-x)=1-\Phi(x)$. We close with Example 12.11, computing $P(-1\le Z\le 1.5)\approx 0.7745$, and record the landmark values $P(-1\le Z\le1)\approx0.683$ and $P(-2\le Z\le2)\approx0.954$.

The expectation (or mean) of a random variable is the single number that says where its probability is centred — a weighted average of the values, each weighted by its probability. For a discrete variable this is $\mathbb{E}[X]=\sum_k k\,\mathbb{P}\{X=k\}$ (Definition 13.1), also called the first moment and written $\mu=\mathbb{E}[X]$; rolling a fair die gives $\mathbb{E}[X]=\tfrac16(1+\cdots+6)=3.5$ (Example 13.1) — already a sign that the mean need not be a value $X$ can actually take. For a continuous variable the sum becomes an integral against the density, $\mathbb{E}[X]=\int_{-\infty}^{\infty} x\,f(x)\,dx$ (Definition 13.2), which for $X\sim\mathrm{Unif}[a,b]$ returns the midpoint $\tfrac{a+b}{2}$ (Example 13.7). To average a function of $X$ you do not need the distribution of $g(X)$: the law of the unconscious statistician (Proposition 13.1) gives $\mathbb{E}[g(X)]=\sum_k g(k)\,\mathbb{P}\{X=k\}$, and the choice $g(x)=x^n$ produces the moments $\mathbb{E}[X^n]$ (Definition 13.3). Expectation is powerful but not automatic: it is well-defined only when its sum or integral settles on a value, finite or $\pm\infty$ (Remark 13.1), and it can be infinite (Examples 13.5, 13.8) or even undefined (Example 13.6). (The means of the named distributions — Bernoulli, binomial, geometric — are developed in a later lesson; here we keep them light.)

The mean locates a random variable; the variance measures how far it typically strays from that centre. For $X$ with mean $\mu=\mathbb{E}[X]$, Definition 13.4 sets $\mathrm{Var}(X)=\mathbb{E}[(X-\mu)^2]$ — the average squared deviation from the mean — written $\sigma^2$, with its square root the standard deviation $\sigma=\mathrm{SD}(X)=\sqrt{\mathrm{Var}(X)}$ restoring the original units. Squaring the deviation inside the expectation makes Proposition 13.2 a sum for a discrete variable, $\mathrm{Var}(X)=\sum_k(k-\mu)^2\,\mathbb{P}(X=k)$, but expanding that square yields the far handier computational formula of Proposition 13.3, $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mu^2$ — the mean of the square minus the square of the mean. From the definition flow the transformation rules of Proposition 13.4: a shift moves the centre but not the spread, while a scale factor comes out squared, $\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)$ (so $\mathrm{SD}(aX+b)=|a|\,\mathrm{SD}(X)$). Finally Proposition 13.5 reads off the extreme case — $\mathrm{Var}(X)=0$ precisely when $X$ is almost surely constant. (The specific variances of the Bernoulli, binomial, uniform and geometric families, Examples 13.9–13.12, are collected in a separate lesson on the moments of named distributions.)

Two numbers summarise any distribution: the mean $\mu=\mathbb{E}[X]$ (Definition 13.1, the first moment), which says where the distribution is centred, and the variance $\sigma^2=\mathrm{Var}(X)=\mathbb{E}[(X-\mu)^2]$ (Definition 13.4), which says how widely it spreads about that centre; its square root $\sigma=\mathrm{SD}(X)$ is the standard deviation. For hand computation the variance is almost always found from the computational formula $\mathrm{Var}(X)=\mathbb{E}[X^2]-(\mathbb{E}[X])^2$ (Proposition 13.3). This lesson is a consolidated reference: it collects, once and for all, the mean and variance of the four distributions you meet most often, each derived in the course's own notes. For the discrete families these are the Bernoulli $\mathrm{Ber}(p)$ with mean $p$ (Example 13.2) and variance $p(1-p)$ (Example 13.9); the binomial $\mathrm{Bin}(n,p)$ with mean $np$ (Example 13.3) and variance $np(1-p)$ (Example 13.10); and the geometric $\mathrm{Geom}(p)$ (support $k\ge 1$, p.m.f. $(1-p)^{k-1}p$) with mean $\frac{1}{p}$ (Example 13.4) and variance $\frac{1-p}{p^2}$ (Example 13.12). For the continuous world we record the uniform $\mathrm{Unif}[a,b]$ with mean $\frac{a+b}{2}$ (Example 13.7) and variance $\frac{(b-a)^2}{12}$ (Example 13.11). Once you can name the family and read off its parameters, every mean and variance is a one-line substitution.

The standard normal $Z\sim N(0,1)$ is the reference bell curve: it has mean $0$ and variance $1$ (Proposition 13.6), and its cumulative distribution function is written $\Phi(z)=\mathbb{P}(Z\le z)$. Every other normal is an affine stretch-and-shift of it. For $\mu\in\mathbb{R}$ and $\sigma>0$, setting $X=\sigma Z+\mu$ produces the general normal distribution $X\sim N(\mu,\sigma^2)$ (Definition 13.5), with mean $\mu$, variance $\sigma^2$, and the bell-shaped density $f(x)=\frac{1}{\sigma\sqrt{2\pi}}\exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right),$ centred at $\mu$ with its spread set by $\sigma$. Affine maps keep you inside the family (Proposition 13.7): if $X\sim N(\mu,\sigma^2)$ then $aX+b\sim N(a\mu+b,\,a^2\sigma^2)$, and the special case $Z=\frac{X-\mu}{\sigma}$ — standardization — turns any normal back into the standard one. That is the computational workhorse: $\mathbb{P}(X\le x)=\Phi\!\left(\frac{x-\mu}{\sigma}\right)$ collapses every normal probability to a single $\Phi$-value read from a standard-normal table or software, as in Example 13.14 ($X\sim N(-3,4)$ gives $\mathbb{P}(X\le-1.7)=\Phi(0.65)\approx0.742$). The same device yields the 68–95–99.7 rule: a normal variable falls within $1$, $2$, or $3$ standard deviations of its mean with probability about $0.683$, $0.954$, and $0.997$ (Example 13.15).

Two distributions govern rare, randomly timed events, and this lesson treats them as a pair. The Poisson distribution (Definition 13.6) counts how many such events occur in a fixed window — calls at a switchboard, decays of a sample, requests to a server — when they happen independently at a constant average rate $\lambda>0$. A Poisson$(\lambda)$ variable takes values $k=0,1,2,\dots$ with mass $\mathbb{P}\{X=k\}=\dfrac{e^{-\lambda}\lambda^{k}}{k!}$; summing the series gives $e^{-\lambda}e^{\lambda}=1$, so it is a genuine p.m.f., and a short computation (Proposition 13.8) shows the single parameter $\lambda$ is both the mean and the variance, $\mathbb{E}[X]=\mathrm{Var}(X)=\lambda$ (Example 13.16: $5$ requests per second give $\mathbb{P}\{X=3\}\approx0.140$). The exponential distribution (Definition 13.7) is the continuous companion that measures the waiting time until the next such event. An Exp$(\lambda)$ variable has density $f(x)=\lambda e^{-\lambda x}$ for $x\ge 0$, tail $\mathbb{P}\{X>t\}=e^{-\lambda t}$, c.d.f. $1-e^{-\lambda t}$, mean $1/\lambda$ and variance $1/\lambda^{2}$ (Examples 13.17–13.19). Its signature feature is the memoryless property (Proposition 13.9): $\mathbb{P}\{X>t+s\mid X>t\}=\mathbb{P}\{X>s\}$ — having already waited tells you nothing about how much longer you must wait (Example 13.20, the turtle and the highway).

For large $n$, a $\mathrm{Bin}(n,p)$ count behaves like a normal with the same mean and variance. That is the Central Limit Theorem at work — and it lets us estimate binomial probabilities with the standard normal.

When successes are rare — large $n$, tiny $p$ — the binomial count is approximately Poisson with mean $\lambda=np$. This is the law of rare events, with a clean error bound.

Two approximations, two regimes. Moderate $p$ with large $np(1-p)$ → normal. Tiny $p$ with small $np^2$ → Poisson. Picking the wrong one can be badly off.

The normal approximation lets us say how close the observed frequency $\hat p=S_n/n$ is to the true $p$ — the idea behind confidence intervals — while the law of large numbers guarantees $\hat p\to p$.

The joint p.m.f. gives the probability of every combination of values of several random variables at once; summing it over one variable recovers the marginal distribution of the other.

Random variables are independent exactly when their joint p.m.f. factors into the product of the marginals — for every combination of values, not just one.

A sequence is exchangeable when reordering it doesn't change its distribution. That symmetry lets you compute positional probabilities — like ‘‘the 23rd card is a spade’’ — as if they were the first.

Expectation of a sum is always the sum of expectations — no independence needed. Variance adds too, but only when the variables are uncorrelated. These give slick proofs of the binomial's mean and variance.

Covariance measures how two variables move together; correlation rescales it to $[-1,1]$. Covariance is bilinear — just like an inner product — which is exactly the bridge back to linear algebra and PCA.