Suppose you want to solve the system of linear equations

\begin{align*} 2x_1 + 3x_2 - x_3 &= 7 \\ x_1 - x_2 + 4x_3 &= 2 \\ 5x_1 + 2x_2 + x_3 &= 11. \end{align*}

You could try substituting and eliminating variables one at a time, but after two eliminations you've lost track of which operations you applied and in what order. Now imagine you have a hundred variables, or a thousand, or that the right-hand side changes and you need to resolve the system. The bookkeeping becomes impossible. What you need is a way to represent, manipulate, and reason about a whole collection of linear equations as a single mathematical object. That object is a matrix.

But matrices are not merely a notation trick. They encode linear maps between vector spaces: the coefficient matrix of the system above describes a linear transformation $T: \mathbb{R}^3 \to \mathbb{R}^3$, and solving the system is equivalent to inverting $T$. Once you see matrices as representations of linear maps, the entire subject opens up — row operations become composition with elementary transformations, the determinant measures how $T$ distorts volume, eigenvalues reveal directions the map stretches without rotating, and matrix decompositions expose the geometric core of $T$. This chapter builds that picture from the ground up.

[example: Encoding a Linear System] Consider the system above. Write the coefficients as a rectangular array and the right-hand side as a column: \begin{align*} A &= \begin{pmatrix} 2 & 3 & -1 \\ 1 & -1 & 4 \\ 5 & 2 & 1 \end{pmatrix}, \qquad b = \begin{pmatrix} 7 \\ 2 \\ 11 \end{pmatrix}. \end{align*} Define the matrix-vector product so that entry $i$ of $Ax$ is $\sum_{j=1}^{3} A_{ij} x_j$. Then the system $2x_1 + 3x_2 - x_3 = 7$, etc., becomes the single equation $Ax = b$. The coefficient matrix $A$ captures all the information in the system; the right-hand side $b$ is separate. Later, if we swap the rows of $A$ and perform the same swap on $b$, the solution does not change — we have just reordered the equations. Row operations on $A$ correspond to multiplying on the left by elementary matrices. [/example]

## Definition

The right level of generality for the definition is a field $\mathbb{F}$, which for us will almost always be $\mathbb{R}$ or $\mathbb{C}$. A matrix is simply a finite rectangular array of elements of $\mathbb{F}$, but the definition must record the dimensions precisely so that operations are well-defined.

[definition: Matrix] Let $\mathbb{F}$ be a field and let $m, n \in \mathbb{N}$. An $m \times n$ matrix over $\mathbb{F}$ is a function \begin{align*} A: \{1, \dots, m\} \times \{1, \dots, n\} &\to \mathbb{F} \\ (i, j) &\mapsto A_{ij}, \end{align*} where $A_{ij}$ is called the $(i,j)$-entry of $A$. We display $A$ as a rectangular array with $m$ rows and $n$ columns: \begin{align*} A &= \begin{pmatrix} A_{11} & A_{12} & \cdots & A_{1n} \\ A_{21} & A_{22} & \cdots & A_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ A_{m1} & A_{m2} & \cdots & A_{mn} \end{pmatrix}. \end{align*} The set of all $m \times n$ matrices over $\mathbb{F}$ is denoted $\mathbb{F}^{m \times n}$. [/definition]

The index convention matters: $A_{ij}$ always means row $i$, column $j$. This is consistent with the notation for the Jacobian matrix $Jf_a \in \mathbb{R}^{n \times m}$ where $(Jf_a)_{ij} = \partial_j f_i(a)$ — the first index labels the output component (row), the second the input component (column). Never use superscripts for matrix entries: write $A_{ij}$, not $A^i_j$ or $A^{ij}$.

[remark: Square Matrices] When $m = n$, we call $A$ a square matrix of size $n$, or an $n \times n$ matrix. Square matrices are the setting for most of the deeper theory: determinants, eigenvalues, and diagonalisation all require $m = n$. For $m \neq n$, the map $x \mapsto Ax$ cannot be invertible simply for dimensional reasons. [/remark]

The set $\mathbb{F}^{m \times n}$ carries the structure of a vector space. Scalar multiplication and addition are defined entry-wise: $(cA)_{ij} = c A_{ij}$ and $(A + B)_{ij} = A_{ij} + B_{ij}$ for $A, B \in \mathbb{F}^{m \times n}$ and $c \in \mathbb{F}$. Since entries are in $\mathbb{F}$ and all operations are inherited from $\mathbb{F}$, all the vector space axioms hold automatically. The zero matrix $0 \in \mathbb{F}^{m \times n}$ has all entries zero, and $\dim \mathbb{F}^{m \times n} = mn$.

The deeper structure, which distinguishes matrices from general vector spaces, is the product. When $A \in \mathbb{F}^{m \times p}$ and $B \in \mathbb{F}^{p \times n}$, their product $AB \in \mathbb{F}^{m \times n}$ is defined by

\begin{align*} (AB)_{ij} &= \sum_{k=1}^{p} A_{ik} B_{kj}. \end{align*}

This formula encodes composition of linear maps: if $A$ represents a map $\mathbb{F}^p \to \mathbb{F}^m$ and $B$ represents a map $\mathbb{F}^n \to \mathbb{F}^p$, then $AB$ represents their composition $\mathbb{F}^n \to \mathbb{F}^m$. The dimension constraint — the number of columns of $A$ must equal the number of rows of $B$ — is exactly the condition that the maps compose.

The problem that the matrix product solves is this: given two linear maps applied in sequence, how do you write the composed map as a single matrix? If you apply $B$ first (a map $\mathbb{F}^n \to \mathbb{F}^p$) and then $A$ (a map $\mathbb{F}^p \to \mathbb{F}^m$), you want a single $m \times n$ matrix $C$ such that $C x = A(Bx)$ for all $x \in \mathbb{F}^n$. The formula $(AB)_{ij} = \sum_k A_{ik} B_{kj}$ is the unique answer: it computes the $i$-th output component of $A(Bx)$ by dotting the $i$-th row of $A$ against the $j$-th column of $B$, which is exactly how composition of linear maps works at the entry level.

[definition: Matrix Product] Let $A \in \mathbb{F}^{m \times p}$ and $B \in \mathbb{F}^{p \times n}$. The matrix product $AB \in \mathbb{F}^{m \times n}$ is defined by \begin{align*} (AB)_{ij} &= \sum_{k=1}^{p} A_{ik} B_{kj}, \qquad 1 \le i \le m, \quad 1 \le j \le n. \end{align*} [/definition]

[remark: Non-Commutativity] Matrix multiplication is not commutative in general. Even for square matrices, $AB = BA$ is a special condition — it says that the corresponding linear maps commute, which is geometrically restrictive. For example, rotation by $90°$ followed by reflection across the $x$-axis gives a different result than reflection first, then rotation. [/remark]

The identity matrix $I_n \in \mathbb{F}^{n \times n}$ has $(I_n)_{ij} = 1$ if $i = j$ and $0$ otherwise. It satisfies $AI_n = A$ and $I_m A = A$ for any $A \in \mathbb{F}^{m \times n}$: $I_n$ is the identity on $\mathbb{F}^n$ (inputs), $I_m$ is the identity on $\mathbb{F}^m$ (outputs). The matrix $I_n$ represents the identity linear map $\operatorname{id}: \mathbb{F}^n \to \mathbb{F}^n$.

[definition: Transpose] Let $A \in \mathbb{F}^{m \times n}$. The transpose of $A$ is the matrix $A^\top \in \mathbb{F}^{n \times m}$ defined by \begin{align*} (A^\top)_{ij} &= A_{ji}. \end{align*} Rows of $A$ become columns of $A^\top$, and columns become rows. [/definition]

The transpose satisfies $(AB)^\top = B^\top A^\top$ — note the reversal of order. This reflects the fact that transposing is adjoint-taking for maps between Euclidean spaces: if $A$ represents $T: \mathbb{R}^n \to \mathbb{R}^m$, then $A^\top$ represents the adjoint $T^*: \mathbb{R}^m \to \mathbb{R}^n$ defined by $\langle Tx, y \rangle = \langle x, T^* y \rangle$.