# Eigenvalue and Eigenvector

A [linear map](/page/Linear%20Map) $T : V \to V$ on a finite-dimensional vector space $V$ over a field $F$ acts on every vector simultaneously, and understanding $T$ in full generality can be daunting. However, a remarkable simplification occurs whenever we can find vectors whose image under $T$ points in the same direction as the original -- vectors that $T$ merely scales. These are the **eigenvectors** of $T$, and the corresponding scale factors are the **eigenvalues**. The theory of eigenvalues and eigenvectors is one of the central pillars of linear algebra, with applications ranging from systems of differential equations to quantum mechanics, from Google's PageRank algorithm to the stability analysis of dynamical systems.

This article develops the theory from its geometric motivation through the characteristic polynomial and multiplicity theory, culminating in the diagonalisation criterion and the spectral theorem for Hermitian matrices.

## Motivation

[motivation] ### The diagonal dream Suppose we are given a linear map $T : V \to V$ on a finite-dimensional vector space $V$ over a field $F$, and suppose we can find a basis $\mathcal{B} = (v_1, v_2, \ldots, v_n)$ of $V$ such that $T$ acts on each basis vector by simply scaling it: \begin{align*} T(v_1) &= \lambda_1 v_1, \\ T(v_2) &= \lambda_2 v_2, \\ &\;\;\vdots \\ T(v_n) &= \lambda_n v_n, \end{align*} for scalars $\lambda_1, \lambda_2, \ldots, \lambda_n \in F$. In this situation the matrix of $T$ with respect to $\mathcal{B}$ is the diagonal matrix $D = \operatorname{diag}(\lambda_1, \ldots, \lambda_n)$. Diagonal matrices are the simplest matrices to work with: computing $D^k$ reduces to raising each diagonal entry to the $k$-th power, solving the system $Dx = b$ reduces to $n$ independent one-variable equations, and the matrix exponential $e^{tD}$ is simply $\operatorname{diag}(e^{\lambda_1 t}, \ldots, e^{\lambda_n t})$. This makes diagonal representations indispensable in applications. For a system of linear ordinary differential equations $x'(t) = Ax(t)$ where $A$ is a real $n \times n$ matrix, if $A$ is diagonalisable with $A = PDP^{-1}$, then the substitution $y = P^{-1}x$ decouples the system into $n$ independent equations $y_i'(t) = \lambda_i y_i(t)$, each with the elementary solution $y_i(t) = y_i(0) e^{\lambda_i t}$. ### What goes wrong Not every linear map admits such a convenient basis. Consider the matrix \begin{align*} A = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} \end{align*} acting on $\mathbb{R}^2$. The only eigenvalue is $\lambda = 0$, but the eigenspace is the span of $(1, 0)^T$, which is one-dimensional. There is no basis of $\mathbb{R}^2$ consisting of eigenvectors of $A$, so $A$ cannot be diagonalised. Such matrices are called **defective**. The central question of this article is: given a linear map $T$, how do we determine whether a basis of eigenvectors exists, and if it does, how do we find it? ### Eigenvectors as the building blocks Geometrically, an eigenvector of $T$ determines an **invariant direction** -- a one-dimensional subspace $\operatorname{span}(v)$ that $T$ maps into itself. The eigenvalue $\lambda$ records whether $T$ stretches ($|\lambda| > 1$), compresses ($|\lambda| < 1$), reflects ($\lambda < 0$), or annihilates ($\lambda = 0$) vectors along that direction. Understanding all the invariant directions of $T$, together with the dimensions of the spaces they generate, is the key to deciding whether $T$ can be diagonalised. [/motivation]

## Formal Definition

Let $V$ be a finite-dimensional vector space over a field $F$, and let $T : V \to V$ be a linear map.

[definition:Eigenvalue And Eigenvector] A scalar $\lambda \in F$ is called an **eigenvalue** of $T$ if there exists a nonzero vector $v \in V$ such that \begin{align*} T(v) = \lambda v. \end{align*} Any nonzero vector $v \in V$ satisfying $T(v) = \lambda v$ is called an **eigenvector** of $T$ corresponding to the eigenvalue $\lambda$. When $T$ is represented by a matrix $A \in M_{n \times n}(F)$ with respect to some basis, we equivalently say that $\lambda$ is an eigenvalue of $A$ and $v \in F^n \setminus \{\mathbf{0}\}$ is an eigenvector of $A$ if $Av = \lambda v$. [/definition]

The requirement that $v \neq \mathbf{0}$ is essential: since $T(\mathbf{0}) = \lambda \cdot \mathbf{0}$ for every $\lambda \in F$, allowing the zero vector would make every scalar an eigenvalue and render the definition useless.

[definition:Eigenspace] Let $\lambda \in F$ be an eigenvalue of $T : V \to V$. The **eigenspace** of $T$ corresponding to $\lambda$ is the [set](/page/Set) \begin{align*} E_\lambda = \ker(T - \lambda I) = \{ v \in V : T(v) = \lambda v \}. \end{align*} This is a subspace of $V$, and its nonzero elements are precisely the eigenvectors corresponding to $\lambda$. [/definition]

The eigenspace $E_\lambda$ is indeed a subspace because it is the kernel of the linear map $T - \lambda I : V \to V$. Note that $\lambda$ is an eigenvalue of $T$ if and only if $E_\lambda \neq \{\mathbf{0}\}$, which happens if and only if $T - \lambda I$ is not injective, which in turn happens if and only if $T - \lambda I$ is singular.

## Examples

[example:TwoByTwoEigenvalues] **Finding eigenvalues of a $2 \times 2$ matrix.** Let \begin{align*} A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix} \in M_{2 \times 2}(\mathbb{R}). \end{align*} To find the eigenvalues we seek $\lambda \in \mathbb{R}$ such that $A - \lambda I$ is singular, i.e., $\det(A - \lambda I) = 0$. We compute \begin{align*} A - \lambda I &= \begin{pmatrix} 4 - \lambda & 1 \\ 2 & 3 - \lambda \end{pmatrix}, \\ \det(A - \lambda I) &= (4 - \lambda)(3 - \lambda) - 2 \\ &= \lambda^2 - 7\lambda + 12 - 2 \\ &= \lambda^2 - 7\lambda + 10 \\ &= (\lambda - 2)(\lambda - 5). \end{align*} So $A$ has eigenvalues $\lambda_1 = 2$ and $\lambda_2 = 5$. For $\lambda_1 = 2$: we solve $(A - 2I)v = \mathbf{0}$: \begin{align*} \begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}, \end{align*} which gives $2v_1 + v_2 = 0$, so $E_2 = \operatorname{span}\!\left(\begin{pmatrix} 1 \\ -2 \end{pmatrix}\right)$. For $\lambda_2 = 5$: we solve $(A - 5I)v = \mathbf{0}$: \begin{align*} \begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}, \end{align*} which gives $-v_1 + v_2 = 0$, so $E_5 = \operatorname{span}\!\left(\begin{pmatrix} 1 \\ 1 \end{pmatrix}\right)$. [/example]

[example:ThreeByThreeEigenvalues] **A $3 \times 3$ matrix with a repeated eigenvalue.** Let \begin{align*} B = \begin{pmatrix} 2 & 1 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 3 \end{pmatrix} \in M_{3 \times 3}(\mathbb{R}). \end{align*} Since $B$ is upper-triangular, its eigenvalues are the diagonal entries: $\lambda_1 = 2$ (repeated) and $\lambda_2 = 3$. More precisely, the characteristic polynomial is \begin{align*} \chi_B(\lambda) = (2 - \lambda)^2(3 - \lambda). \end{align*} For $\lambda_1 = 2$: \begin{align*} B - 2I = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix}. \end{align*} The kernel has dimension $1$ (only the first coordinate is free), giving $E_2 = \operatorname{span}\!\left(\begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}\right)$. Even though $\lambda_1 = 2$ appears as a root of multiplicity $2$ in the characteristic polynomial, the eigenspace is only one-dimensional. This discrepancy between algebraic and geometric multiplicity will be the focus of a later section. For $\lambda_2 = 3$: $E_3 = \operatorname{span}\!\left(\begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}\right)$, which is one-dimensional. [/example]

## The Characteristic Polynomial

The examples above suggest a systematic approach: $\lambda$ is an eigenvalue of $A$ if and only if $\det(A - \lambda I) = 0$. The expression $\det(A - \lambda I)$, viewed as a function of $\lambda$, is a polynomial, and its roots are precisely the eigenvalues.

[definition:Characteristic Polynomial] Let $A \in M_{n \times n}(F)$. The **characteristic polynomial** of $A$ is the polynomial $\chi_A \in F[\lambda]$ defined by \begin{align*} \chi_A(\lambda) = \det(A - \lambda I). \end{align*} This is a monic polynomial of degree $n$ in $\lambda$ (up to sign convention; some authors define $\chi_A(\lambda) = \det(\lambda I - A)$, which gives the same roots but a different leading coefficient). [/definition]

The characteristic polynomial is well-defined as an invariant of the linear map $T$ rather than of any particular matrix representation: if $B = P^{-1}AP$ for some invertible $P$, then

\begin{align*} \det(B - \lambda I) = \det(P^{-1}AP - \lambda P^{-1}P) = \det(P^{-1}(A - \lambda I)P) = \det(A - \lambda I), \end{align*}

so similar matrices share the same characteristic polynomial. We may therefore speak of the characteristic polynomial $\chi_T$ of a linear map $T$ without ambiguity.