Matrix Operations

What a matrix is. A matrix is a rectangular grid of numbers. We say a matrix is $m \times n$ (read "$m$ by $n$") when it has $m$ rows and $n$ columns; the pair $(m,n)$ is its shape (or dimension). We write $A=(a_{ij})$, where the entry $a_{ij}$ is the number in row $i$, column $j$ (rows and columns are numbered starting at $1$). For example, $A=\begin{bmatrix} 1 & 2 & 0 \\ 3 & -1 & 4 \end{bmatrix}$ is $2\times 3$, and $a_{23}=4$.

Addition (same shape, entrywise). You may add two matrices only when they have the same shape. Then $A+B$ is formed by adding corresponding entries: $(A+B)_{ij}=a_{ij}+b_{ij}.$ Subtraction works the same way. If the shapes differ, $A+B$ is simply undefined.

Scalar multiplication (entrywise). A scalar is just a number $c$. The scalar multiple $cA$ scales every entry: $(cA)_{ij}=c\,a_{ij}.$ For instance, $2\begin{bmatrix} 1 & -3 \\ 0 & 5 \end{bmatrix}=\begin{bmatrix} 2 & -6 \\ 0 & 10 \end{bmatrix}$.

Useful facts. Addition is commutative and associative: $A+B=B+A$ and $(A+B)+C=A+(B+C)$. Scalars distribute over sums: $c(A+B)=cA+cB$. These all hold because every rule is applied one entry at a time, where ordinary number arithmetic already obeys them.

When is $AB$ defined? Unlike addition, multiplication does not require equal shapes. Instead it uses the dimension rule: if $A$ is $m\times n$ and $B$ is $n\times p$ — that is, the number of columns of $A$ equals the number of rows of $B$ — then $AB$ is defined and has shape $m\times p$: $(\underbrace{m\times n}_{A})\,(\underbrace{n\times p}_{B})=\underbrace{m\times p}_{AB}.$ The two inner numbers must match; the two outer numbers give the result's shape. If the inner numbers disagree, $AB$ is undefined.

The row-by-column rule. The entry in row $i$, column $j$ of $AB$ is the dot product of row $i$ of $A$ with column $j$ of $B$: $(AB)_{ij}=\sum_{k=1}^{n} a_{ik}\,b_{kj}=a_{i1}b_{1j}+a_{i2}b_{2j}+\cdots+a_{in}b_{nj}.$ Here the index $k$ runs across the shared dimension $n$. Concretely, to get the top-left entry you slide row $1$ of $A$ across column $1$ of $B$, multiply matching pairs, and add them up.

A quick computation. With $A=\begin{bmatrix} 1 & 2 \\ 0 & 3 \end{bmatrix},\qquad B=\begin{bmatrix} 4 & 1 \\ 2 & 5 \end{bmatrix},$ the $(1,1)$ entry is $1\cdot 4 + 2\cdot 2 = 8$ and the $(2,2)$ entry is $0\cdot 1 + 3\cdot 5 = 15$. Filling in all four entries, $AB=\begin{bmatrix} 8 & 11 \\ 6 & 15 \end{bmatrix}.$

Order matters. For ordinary numbers $xy=yx$, but for matrices $AB$ and $BA$ are usually different — matrix multiplication is non-commutative. Sometimes only one of the two products is even defined (the dimension rule can fail in the opposite order), and sometimes both are defined but give different results or even different shapes. For example, with $A=\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix},\qquad B=\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix},$ we get $AB=\begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix}$ while $BA=\begin{bmatrix} 3 & 4 \\ 1 & 2 \end{bmatrix}$. Since $AB\neq BA$, the blanket claim "$AB=BA$" is false in general.

The identity matrix. The $n\times n$ identity matrix $I_n$ has $1$s on the main diagonal and $0$s everywhere else, e.g. $I_2=\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$. It behaves like the number $1$ for multiplication: for every $m\times n$ matrix $A$, $A\,I_n=A\qquad\text{and}\qquad I_m\,A=A.$ So multiplying by $I$ leaves a matrix unchanged — one of the few places where matrix multiplication is pleasantly simple.