Orthonormal Bases & Coordinates

A set $\{\mathbf{v}_1,\dots,\mathbf{v}_m\}$ in $\mathbb{R}^n$ is orthogonal when every pair of distinct vectors is perpendicular: $\mathbf{v}_i\cdot\mathbf{v}_j=0\quad\text{for all }i\neq j.$ If in addition each vector has unit length, $\|\mathbf{v}_i\|=1$, the set is orthonormal (Definition 8.7). The two requirements collapse into one tidy formula using the Kronecker delta: $\{\mathbf{u}_1,\dots,\mathbf{u}_m\}$ is orthonormal exactly when $\mathbf{u}_i\cdot\mathbf{u}_j=\delta_{ij}=\begin{cases}1,&i=j,\\0,&i\neq j.\end{cases}$ The diagonal case $\mathbf{u}_i\cdot\mathbf{u}_i=\|\mathbf{u}_i\|^2=1$ is the normal (unit-length) part; the off-diagonal case is the ortho (perpendicular) part. So orthonormal = orthogonal + normalized. The standard basis $\mathbf{e}_1,\dots,\mathbf{e}_n$ is the cleanest example: $\mathbf{e}_i\cdot\mathbf{e}_j=0$ for $i\neq j$ and $\|\mathbf{e}_i\|=1$ (Example 8.3).

Checking linear independence usually means row-reducing a matrix. For orthonormal vectors you get it for free (Proposition 8.1). Suppose $c_1\mathbf{u}_1+\cdots+c_k\mathbf{u}_k=\mathbf{0}.$ Take the dot product of both sides with a fixed $\mathbf{u}_i$. On the right, $\mathbf{0}\cdot\mathbf{u}_i=0$. On the left, every term $c_j(\mathbf{u}_j\cdot\mathbf{u}_i)$ with $j\neq i$ vanishes by orthogonality, and the one surviving term is $c_i(\mathbf{u}_i\cdot\mathbf{u}_i)=c_i\|\mathbf{u}_i\|^2=c_i$. Hence $c_i=0$, and since $i$ was arbitrary, all coefficients are zero. The only linear relation is the trivial one, so the vectors are linearly independent. This same dot-with-$\mathbf{u}_i$ trick is the engine behind the coordinate formula in Theorem 8.5.

Because orthonormal vectors are independent, they are a basis of whatever they span. Theorem 8.4: if $\mathbf{u}_1,\dots,\mathbf{u}_k\in\mathbb{R}^n$ are orthonormal, they form a basis for their span $V=\operatorname{span}\{\mathbf{u}_1,\dots,\mathbf{u}_k\}$, which is therefore a $k$-dimensional subspace of $\mathbb{R}^n$; in particular, $n$ orthonormal vectors in $\mathbb{R}^n$ form a basis of all of $\mathbb{R}^n$. Remark 8.4 extends this: Proposition 8.1 and Theorem 8.4 hold for nonzero orthogonal vectors too — the word nonzero matters, since $\mathbf{0}$ is orthogonal to everything yet destroys independence. And any orthogonal basis $\{\mathbf{v}_1,\dots,\mathbf{v}_n\}$ is made orthonormal by normalizing each vector, $\mathbf{u}_i=\frac{\mathbf{v}_i}{\|\mathbf{v}_i\|}.$ Example 8.4 does exactly this with $(1,2,-1),(0,1,2),(5,-2,1)$.

In a general basis, writing $\mathbf{v}=c_1\mathbf{u}_1+\cdots+c_n\mathbf{u}_n$ means solving an $n\times n$ linear system for the $c_i$. In an orthonormal basis there is nothing to solve. Theorem 8.5: for any $\mathbf{v}\in\mathbb{R}^n$, $\mathbf{v}=\sum_{i=1}^n c_i\mathbf{u}_i,\qquad c_i=\mathbf{v}\cdot\mathbf{u}_i,$ and this representation is unique. The proof is the Proposition 8.1 trick once more: dot $\mathbf{v}=\sum_j c_j\mathbf{u}_j$ with $\mathbf{u}_i$ and only $c_i$ survives. Each coordinate is a single dot product. (For a merely orthogonal basis, combine with Remark 8.4 to get $c_i=\dfrac{\mathbf{v}\cdot\mathbf{v}_i}{\|\mathbf{v}_i\|^2}$.) This efficiency is exactly why orthonormal bases power Fourier analysis, least-squares approximation, and large-scale data methods.