[proofplan]
We prove both parts of the theorem using the linearisation $A = Jf_{x^*}$ and the Grönwall inequality. For asymptotic stability (all eigenvalues in the open left half-plane), we construct a Lyapunov function from a quadratic form and show that trajectories decay exponentially. For instability (at least one eigenvalue with positive real part), we show that the unstable subspace generates exponential growth that dominates the nonlinear corrections. Both arguments exploit the fact that $f(x) = A(x - x^*) + O(|x - x^*|^2)$ near $x^*$.
[/proofplan]
[step:Translate to the origin and decompose the vector field]
Without loss of generality, translate coordinates so that $x^* = 0$. Since $f$ is $C^1$ with $f(0) = 0$, Taylor expansion gives
\begin{align*}
f(x) = Ax + g(x),
\end{align*}
where $A := Jf_0$ and $g: U \to \mathbb{R}^n$ satisfies $g(0) = 0$ and $|g(x)| / |x| \to 0$ as $x \to 0$. For any $\varepsilon > 0$, there exists $\delta > 0$ such that $|g(x)| \leq \varepsilon |x|$ for all $|x| \leq \delta$.
[/step]
[step:Prove asymptotic stability when all eigenvalues satisfy $\operatorname{Re}(\lambda_i) < 0$]
Suppose $\operatorname{Re}(\lambda_i) < 0$ for all $i = 1, \ldots, n$. Choose $\alpha > 0$ such that $\operatorname{Re}(\lambda_i) \leq -\alpha$ for all $i$. By the theory of matrix exponentials (using a norm adapted to the Jordan structure of $A$ if necessary), there exists a constant $C \geq 1$ such that
\begin{align*}
|e^{At}| \leq C e^{-\alpha t} \quad \text{for all } t \geq 0.
\end{align*}
By the variation-of-constants formula, the solution of $\dot{x} = Ax + g(x)$ with $x(0) = x_0$ satisfies
\begin{align*}
x(t) = e^{At} x_0 + \int_0^t e^{A(t - \tau)} g(x(\tau)) \, d\mathcal{L}^1(\tau).
\end{align*}
Taking norms and using the bound $|g(x)| \leq \varepsilon |x|$ for $|x| \leq \delta$:
\begin{align*}
|x(t)| \leq C e^{-\alpha t} |x_0| + \int_0^t C e^{-\alpha(t - \tau)} \varepsilon |x(\tau)| \, d\mathcal{L}^1(\tau),
\end{align*}
provided $|x(\tau)| \leq \delta$ for all $\tau \in [0, t]$. Define $\psi(t) := e^{\alpha t} |x(t)|$. Then
\begin{align*}
\psi(t) \leq C |x_0| + C\varepsilon \int_0^t \psi(\tau) \, d\mathcal{L}^1(\tau).
\end{align*}
By the [Grönwall Inequality](/theorems/???), $\psi(t) \leq C|x_0| \, e^{C\varepsilon t}$, hence
\begin{align*}
|x(t)| \leq C |x_0| \, e^{(-\alpha + C\varepsilon) t}.
\end{align*}
Choose $\varepsilon < \alpha / C$. Then $-\alpha + C\varepsilon < 0$, and $|x(t)|$ decays exponentially. Choose $\delta$ corresponding to this $\varepsilon$, and take $|x_0| < \delta / C$ to ensure $|x(t)| \leq C|x_0| < \delta$ for all $t \geq 0$ (the exponential decay guarantees the trajectory stays in the ball where the estimate is valid). In particular, $|x(t)| \to 0$ as $t \to +\infty$, proving asymptotic stability.
[guided]
Suppose all eigenvalues of $A = Jf_0$ satisfy $\operatorname{Re}(\lambda_i) < 0$. Choose $\alpha > 0$ with $\operatorname{Re}(\lambda_i) \leq -\alpha$ for all $i$. By the theory of matrix exponentials (using a norm adapted to the Jordan structure of $A$), there exists $C \geq 1$ such that $|e^{At}| \leq Ce^{-\alpha t}$ for all $t \geq 0$.
The solution of $\dot{x} = Ax + g(x)$ with $x(0) = x_0$ is given by the variation-of-constants formula:
\begin{align*}
x(t) = e^{At} x_0 + \int_0^t e^{A(t-\tau)} g(x(\tau)) \, d\mathcal{L}^1(\tau).
\end{align*}
Taking norms and using $|g(x)| \leq \varepsilon |x|$ for $|x| \leq \delta$ (which holds by choosing $\delta$ small since $|g(x)|/|x| \to 0$):
\begin{align*}
|x(t)| \leq Ce^{-\alpha t} |x_0| + \int_0^t Ce^{-\alpha(t-\tau)} \varepsilon |x(\tau)| \, d\mathcal{L}^1(\tau).
\end{align*}
Now define $\psi(t) := e^{\alpha t} |x(t)|$. Multiplying both sides by $e^{\alpha t}$ gives $\psi(t) \leq C|x_0| + C\varepsilon \int_0^t \psi(\tau) \, d\mathcal{L}^1(\tau)$. By the [Gronwall Inequality](/theorems/???), $\psi(t) \leq C|x_0| e^{C\varepsilon t}$, hence
\begin{align*}
|x(t)| \leq C|x_0| \, e^{(-\alpha + C\varepsilon)t}.
\end{align*}
The argument now closes by a bootstrap. Choose $\varepsilon < \alpha/C$, so that the exponent $-\alpha + C\varepsilon < 0$. Let $\delta$ be the radius corresponding to this $\varepsilon$ (i.e., $|g(x)| \leq \varepsilon |x|$ for $|x| \leq \delta$). Take $|x_0| < \delta/C$. Then $|x(t)| \leq C|x_0| e^{(-\alpha + C\varepsilon)t} \leq C|x_0| < \delta$ for all $t \geq 0$, since the exponential factor is at most $1$ (its exponent is negative). So the trajectory never leaves $B(0, \delta)$, the estimate $|g(x)| \leq \varepsilon|x|$ remains valid, and the bound is self-consistent.
Why does the nonlinear term not destroy stability? Because near the fixed point, $g(x)$ is much smaller than $Ax$ -- the ratio $|g(x)|/|x| \to 0$ as $x \to 0$. The linear part decays at rate $\alpha$, and the nonlinear perturbation can only slow this decay by at most $C\varepsilon$. By choosing $\varepsilon$ small enough, the net rate $\alpha - C\varepsilon$ remains positive. This is why hyperbolicity (a spectral gap between the eigenvalues and the imaginary axis) is essential: it provides the margin $\alpha$ that absorbs the nonlinear perturbation.
[/guided]
[/step]
[step:Prove instability when some eigenvalue satisfies $\operatorname{Re}(\lambda_i) > 0$]
Suppose $\operatorname{Re}(\lambda_j) > 0$ for some $j$. Let $E^u$ be the direct sum of generalised eigenspaces for eigenvalues with positive real part, with $\dim E^u = n^u \geq 1$, and let $\pi_u: \mathbb{R}^n \to E^u$ be the projection along the stable subspace $E^s$. Choose $\beta > 0$ such that $\operatorname{Re}(\lambda) \geq \beta$ for all eigenvalues of $A|_{E^u}$. There exists $C' > 0$ such that
\begin{align*}
|e^{A_u t} v| \geq \frac{1}{C'} e^{\beta t} |v| \quad \text{for all } v \in E^u, \; t \geq 0,
\end{align*}
where $A_u := A|_{E^u}$.
For any $x_0 \in E^u \setminus \{0\}$ with $|x_0|$ small, write $x(t) = \varphi(t, x_0)$ and consider $x_u(t) := \pi_u(x(t))$. By the variation-of-constants formula projected onto $E^u$:
\begin{align*}
x_u(t) = e^{A_u t} \pi_u(x_0) + \int_0^t e^{A_u(t - \tau)} \pi_u(g(x(\tau))) \, d\mathcal{L}^1(\tau).
\end{align*}
Since $\pi_u(x_0) = x_0$ (as $x_0 \in E^u$), the leading term grows as $|e^{A_u t} x_0| \geq e^{\beta t} |x_0| / C'$. The integral term is bounded by $\int_0^t C'' e^{\beta(t-\tau)} \varepsilon |x(\tau)| \, d\mathcal{L}^1(\tau)$ for some constant $C''$, which is of smaller order when $\varepsilon$ is small and $|x|$ remains controlled. A Grönwall-type argument shows that for $\varepsilon$ sufficiently small, the unstable component satisfies
\begin{align*}
|x_u(t)| \geq \frac{1}{2C'} e^{\beta t / 2} |x_0|
\end{align*}
for $t$ in an interval $[0, T_{x_0}]$ where $|x(t)| \leq \delta$. Since $|x_u(t)| \leq |x(t)|$, this forces $|x(t)| \geq \frac{1}{2C'} e^{\beta t/2} |x_0|$, which reaches $\delta$ in finite time. Hence the trajectory escapes every small neighbourhood of $0$, proving instability.
[guided]
The instability argument is the converse of the stability argument: we show that the linear expansion along the unstable eigenspace $E^u$ dominates the nonlinear corrections, forcing trajectories to escape every neighbourhood of $0$.
Suppose $\operatorname{Re}(\lambda_j) > 0$ for some eigenvalue of $A$. Let $E^u$ be the generalised eigenspace for eigenvalues with positive real part, and let $\pi_u: \mathbb{R}^n \to E^u$ be the projection. Choose $\beta > 0$ with $\operatorname{Re}(\lambda) \geq \beta$ for all eigenvalues of $A_u := A|_{E^u}$. There exists $C' > 0$ such that $|e^{A_u t} v| \geq (1/C') e^{\beta t} |v|$ for all $v \in E^u$ and $t \geq 0$.
For $x_0 \in E^u \setminus \{0\}$ with $|x_0|$ small, the variation-of-constants formula projected onto $E^u$ gives:
\begin{align*}
x_u(t) = e^{A_u t} x_0 + \int_0^t e^{A_u(t-\tau)} \pi_u(g(x(\tau))) \, d\mathcal{L}^1(\tau).
\end{align*}
The leading term satisfies $|e^{A_u t} x_0| \geq (1/C') e^{\beta t} |x_0|$, which grows exponentially. The integral correction is bounded by $\int_0^t C'' e^{\beta(t-\tau)} \varepsilon |x(\tau)| \, d\mathcal{L}^1(\tau)$, where $\varepsilon$ is the Lipschitz constant of $g$ in the ball $B(0, \delta)$.
A Gronwall-type lower bound argument shows that for $\varepsilon$ sufficiently small (specifically, $\varepsilon \ll \beta$), the growth of the leading term dominates the correction, yielding
\begin{align*}
|x_u(t)| \geq \frac{1}{2C'} e^{\beta t/2} |x_0|
\end{align*}
for all $t \in [0, T_{x_0}]$ where $|x(t)| \leq \delta$. Since $|x(t)| \geq |x_u(t)| \geq (1/2C') e^{\beta t/2} |x_0|$, this exponential growth eventually pushes $|x(t)|$ past $\delta$ in finite time $T_{x_0} \leq (2/\beta) \ln(2C'\delta/|x_0|)$.
Why do we restrict to initial conditions in $E^u$? Because instability requires only that *some* arbitrarily small initial conditions escape -- not all. Trajectories starting in $E^s$ may indeed decay to $0$. But the existence of escaping trajectories in every neighbourhood of $0$ is exactly what instability means (the fixed point fails to be Lyapunov stable). Choosing $x_0 \in E^u \setminus \{0\}$ guarantees that the growing linear mode is excited and cannot be cancelled by the stable component.
[/guided]
[/step]