[proofplan] We translate the equilibrium to the origin and write the vector field as its linear part plus a remainder term that is smaller than linear near the origin. In the stable case, the spectral hypothesis on the linear part gives a positive definite quadratic Lyapunov function by solving the Lyapunov equation through a convergent matrix integral. The nonlinear remainder is absorbed by choosing the neighbourhood small enough, giving exponential decay. In the unstable case, the local center-[stable manifold theorem](/theorems/2778) shows that all sufficiently small initial data whose forward orbits remain near the equilibrium lie in a proper lower-dimensional manifold; hence arbitrarily small initial data outside that manifold must leave a fixed neighbourhood. [/proofplan]

[step:Translate the equilibrium to the origin and isolate the nonlinear remainder] Define the translated [open set](/page/Open%20Set) $V \subset \mathbb{R}^n$ by \begin{align*} V := U - x^* = \{y \in \mathbb{R}^n : y + x^* \in U\}. \end{align*} Define the translated vector field $Y: V \to \mathbb{R}^n$ by \begin{align*} Y(y) := X(y + x^*). \end{align*} Then $Y \in C^1(V;\mathbb{R}^n)$ and $Y(0) = 0$. Define the matrix $A \in \mathbb{R}^{n \times n}$ by \begin{align*} A := JY_0. \end{align*} Since translation does not change the Jacobian matrix, \begin{align*} A = JX_{x^*}. \end{align*} Define the nonlinear remainder $R: V \to \mathbb{R}^n$ by \begin{align*} R(y) := Y(y) - Ay. \end{align*} Because $Y$ is differentiable at $0$, $Y(0) = 0$, and $JY_0 = A$, the remainder satisfies \begin{align*} \lim_{y \to 0,\, y \ne 0} \frac{|R(y)|}{|y|} = 0. \end{align*} Thus the translated system is \begin{align*} \dot{y} = Ay + R(y). \end{align*} Asymptotic stability or Lyapunov instability of $0$ for this translated system is equivalent to the corresponding property of $x^*$ for the original system under the translation $x = y + x^*$. [/step]

[step:Construct a quadratic Lyapunov function when the linearization is Hurwitz]Let $\mathcal{L}^1$ denote one-dimensional [Lebesgue measure](/page/Lebesgue%20Measure) on $\mathbb{R}$. Assume every eigenvalue $\lambda$ of $A$ satisfies $\operatorname{Re}(\lambda) < 0$. For a [linear map](/page/Linear%20Map) $B$ on a Euclidean space, write its operator norm as \begin{align*} \|B\|_{\mathrm{op}}:=\sup\{|Bv|:|v|=1\}. \end{align*} [claim:The linear flow decays exponentially] There exist constants $M \ge 1$ and $\alpha > 0$ such that \begin{align*} \|e^{tA}\|_{\mathrm{op}} \le M e^{-\alpha t} \end{align*} for every $t \ge 0$. [/claim] [proof] Complexify $A$ to a complex linear map on $\mathbb{C}^n$. Let $GL(n,\mathbb{C})$ denote the group of invertible complex $n \times n$ matrices. By the [Jordan normal form](/theorems/864) over $\mathbb{C}$, there is an invertible complex matrix $S \in GL(n,\mathbb{C})$ and a Jordan matrix $J \in \mathbb{C}^{n \times n}$ such that $A = SJS^{-1}$ after complexification. If $\lambda_1,\dots,\lambda_m$ are the eigenvalues of $A$, choose $\alpha > 0$ so that $\operatorname{Re}(\lambda_k) \le -2\alpha$ for every $k$. For each Jordan block $J_k = \lambda_k I + N_k$, where $N_k$ is nilpotent, the exponential is \begin{align*} e^{tJ_k} = e^{t\lambda_k} \sum_{j=0}^{d_k-1} \frac{t^j N_k^j}{j!}, \end{align*} where $d_k$ is the size of the block. Since each polynomial factor is bounded by a constant times $e^{\alpha t}$ on $[0,\infty)$, there is $C_J > 0$ such that \begin{align*} \|e^{tJ}\|_{\mathrm{op}} \le C_J e^{-\alpha t} \end{align*} for every $t \ge 0$. Hence \begin{align*} \|e^{tA}\|_{\mathrm{op}} \le \|S\|_{\mathrm{op}} \|S^{-1}\|_{\mathrm{op}} C_J e^{-\alpha t}. \end{align*} Taking \begin{align*} M := \max\{1,\|S\|_{\mathrm{op}}\|S^{-1}\|_{\mathrm{op}}C_J\} \end{align*} proves the claim for the real operator norm by restricting back to $\mathbb{R}^n$. [/proof] Define $P: \mathbb{R}^n \to \mathbb{R}^n$ to be the linear operator determined by the [bilinear form](/page/Bilinear%20Form) \begin{align*} u \cdot Pv := \int_0^\infty (e^{tA}u) \cdot (e^{tA}v)\,d\mathcal{L}^1(t) \end{align*} for $u,v \in \mathbb{R}^n$. The exponential decay estimate makes the integral finite, and the bilinear form is symmetric and positive definite. Therefore $P$ is represented by a real symmetric positive definite matrix. For $v \in \mathbb{R}^n$, differentiate the map $t \mapsto |e^{tA}v|^2$. Since \begin{align*} \frac{d}{dt}|e^{tA}v|^2 = (Ae^{tA}v)\cdot e^{tA}v + e^{tA}v\cdot (Ae^{tA}v), \end{align*} integrating over $[0,T]$ with respect to $\mathcal{L}^1$ gives \begin{align*} |\exp(TA)v|^2 - |v|^2 = \int_{[0,T]} \left((Ae^{tA}v)\cdot e^{tA}v + e^{tA}v\cdot (Ae^{tA}v)\right)\,d\mathcal{L}^1(t). \end{align*} The exponential decay estimate gives $\exp(TA)v \to 0$ and bounds the integrand by a constant multiple of $e^{-2\alpha t}|v|^2$, which is integrable on $[0,\infty)$. Passing to the limit $T \to \infty$ therefore gives \begin{align*} v \cdot (A^\top P + PA)v = -|v|^2. \end{align*} Thus $A^\top P + PA = -I$. Define the quadratic function $L: \mathbb{R}^n \to \mathbb{R}$ by \begin{align*} L(y) := y \cdot Py. \end{align*} Let $m_P > 0$ be the smallest eigenvalue of $P$ and let $M_P := \|P\|_{\mathrm{op}}$. Then \begin{align*} m_P |y|^2 \le L(y) \le M_P |y|^2 \end{align*} for every $y \in \mathbb{R}^n$.[/step]

[guided]The stable linear system $\dot{y} = Ay$ should have a quadratic Lyapunov function. The spectral hypothesis says every mode of the linear flow decays, and the exponential estimate makes it possible to average the Euclidean energy along the whole future orbit. This is why we define $P$ by \begin{align*} u \cdot Pv := \int_0^\infty (e^{tA}u) \cdot (e^{tA}v)\,d\mathcal{L}^1(t). \end{align*} The integral is finite because $\|e^{tA}\|_{\mathrm{op}} \le M e^{-\alpha t}$, so the integrand is bounded by $M^2 e^{-2\alpha t}|u||v|$, whose integral over $[0,\infty)$ with respect to $\mathcal{L}^1$ is finite. The form is symmetric because the Euclidean [inner product](/page/Inner%20Product) is symmetric. It is positive definite because, for $v \ne 0$, \begin{align*} v \cdot Pv = \int_0^\infty |e^{tA}v|^2\,d\mathcal{L}^1(t) > 0. \end{align*} The strict positivity follows from invertibility of $e^{tA}$ for each $t$, so $e^{tA}v \ne 0$ whenever $v \ne 0$. Now we compute the Lyapunov equation. For $v \in \mathbb{R}^n$, the function $t \mapsto |e^{tA}v|^2$ is continuously differentiable, and its derivative is \begin{align*} \frac{d}{dt}|e^{tA}v|^2 = (Ae^{tA}v)\cdot e^{tA}v + e^{tA}v\cdot (Ae^{tA}v). \end{align*} Integrating this identity over $[0,T]$ gives the boundary term \begin{align*} |\exp(TA)v|^2 - |v|^2. \end{align*} The exponential decay estimate implies $\exp(TA)v \to 0$. It also bounds the derivative by a constant multiple of $e^{-2\alpha t}|v|^2$, so the derivative is integrable on $[0,\infty)$ with respect to $\mathcal{L}^1$. Passing to the limit $T \to \infty$ gives \begin{align*} \lim_{T\to\infty}|\exp(TA)v|^2 - |v|^2 = -|v|^2. \end{align*} On the other hand, by the definition of $P$, the same improper integral equals \begin{align*} v \cdot (A^\top P + PA)v. \end{align*} Hence \begin{align*} v \cdot (A^\top P + PA)v = -|v|^2 \end{align*} for every $v \in \mathbb{R}^n$, which gives $A^\top P + PA = -I$. Finally, since $P$ is symmetric positive definite, its eigenvalues are positive. If $m_P$ is the smallest eigenvalue and $M_P = \|P\|_{\mathrm{op}}$ is the largest eigenvalue, then \begin{align*} m_P |y|^2 \le y \cdot Py \le M_P |y|^2. \end{align*} Thus $L(y) = y \cdot Py$ is exactly comparable to the squared Euclidean distance to the equilibrium.[/guided]

[step:Absorb the nonlinear remainder and prove asymptotic stability]Since $V$ is open and contains $0$, choose $r_1 > 0$ such that $\overline{B}(0,r_1) \subset V$. Since $R(y) = o(|y|)$ as $y \to 0$, shrink this radius if necessary and choose $r_0 \in (0,r_1]$ such that \begin{align*} |R(y)| \le \frac{1}{4M_P}|y| \end{align*} whenever $|y| < r_0$. Let $y: [0,T) \to V$ be a solution of $\dot{y} = Ay + R(y)$ with $|y(t)| < r_0$. Along this solution, \begin{align*} \frac{d}{dt}L(y(t)) = y(t)\cdot (A^\top P + PA)y(t) + 2y(t)\cdot P R(y(t)). \end{align*} Using $A^\top P + PA = -I$, the [Cauchy-Schwarz inequality](/theorems/432) in $\mathbb{R}^n$, the operator norm bound $|PR(y)| \le M_P |R(y)|$, and the smallness of $R$, we obtain \begin{align*} \frac{d}{dt}L(y(t)) \le -|y(t)|^2 + 2M_P|y(t)||R(y(t))| \le -\frac{1}{2}|y(t)|^2. \end{align*} Since $L(y) \le M_P |y|^2$, this implies \begin{align*} \frac{d}{dt}L(y(t)) \le -\frac{1}{2M_P}L(y(t)). \end{align*} Choose an arbitrary stability radius $\varepsilon_* > 0$ and define \begin{align*} \varepsilon := \min\{\varepsilon_*, r_0/2\}. \end{align*} Let $\delta > 0$ satisfy \begin{align*} \delta < \sqrt{\frac{m_P}{M_P}}\,\varepsilon. \end{align*} If $|y(0)| < \delta$, then \begin{align*} L(y(0)) \le M_P\delta^2 < m_P\varepsilon^2. \end{align*} As long as the solution remains in $B(0,r_0)$, $L$ is nonincreasing. Therefore the solution cannot reach the sphere $|y|=\varepsilon$, because on that sphere $L(y) \ge m_P\varepsilon^2$. Since $B(0,\varepsilon) \subset B(0,\varepsilon_*)$, this proves Lyapunov stability of $0$ for every prescribed neighbourhood radius $\varepsilon_* > 0$. Taking $\varepsilon < r_0$ and applying Gronwall's inequality to the differential inequality for $L$ gives \begin{align*} L(y(t)) \le L(y(0))\exp\left(-\frac{t}{2M_P}\right) \end{align*} for all $t \ge 0$ for which the solution is defined. The preceding stability argument keeps the solution in $B(0,r_0)$ for all positive time. Since $\overline{B}(0,r_0) \subset V$ is compact and $Y \in C^1(V;\mathbb{R}^n)$ is locally Lipschitz and bounded on a neighbourhood of $\overline{B}(0,r_0)$, the ODE continuation theorem prevents finite positive-time blow-up while the trajectory remains in this compact subset of $V$. Hence the solution is global forward in time. Since $m_P|y(t)|^2 \le L(y(t))$, we conclude that $y(t) \to 0$ as $t \to \infty$. Thus $0$ is asymptotically stable for the translated system, and therefore $x^*$ is asymptotically stable for the original system.[/step]

[guided]The nonlinear term has to be small enough that the strict decrease from the linear Lyapunov equation still dominates. Since $R(y) = o(|y|)$ as $y \to 0$, choose $r_0 > 0$ with $\overline{B}(0,r_0) \subset V$ and \begin{align*} |R(y)| \le \frac{1}{4M_P}|y| \end{align*} whenever $|y| < r_0$. Let $y: [0,T) \to V$ be a solution of $\dot{y}=Ay+R(y)$ whose image remains in $B(0,r_0)$. Differentiating $L(y)=y\cdot Py$ along the solution gives \begin{align*} \frac{d}{dt}L(y(t)) = y(t)\cdot (A^\top P+PA)y(t)+2y(t)\cdot PR(y(t)). \end{align*} The Lyapunov equation gives $A^\top P+PA=-I$. Applying the Cauchy-Schwarz inequality in $\mathbb{R}^n$ and the operator norm bound $|PR(y)| \le M_P|R(y)|$, we obtain \begin{align*} \frac{d}{dt}L(y(t)) \le -|y(t)|^2+2M_P|y(t)||R(y(t))| \le -\frac{1}{2}|y(t)|^2. \end{align*} Because $L(y)\le M_P|y|^2$, this implies \begin{align*} \frac{d}{dt}L(y(t)) \le -\frac{1}{2M_P}L(y(t)). \end{align*} For stability, fix $\varepsilon_* > 0$ and set $\varepsilon := \min\{\varepsilon_*,r_0/2\}$. Choose $\delta > 0$ satisfying \begin{align*} \delta < \sqrt{\frac{m_P}{M_P}}\,\varepsilon. \end{align*} If $|y(0)|<\delta$, then $L(y(0))\le M_P\delta^2<m_P\varepsilon^2$. While the solution remains in $B(0,r_0)$, the preceding differential inequality makes $L$ nonincreasing. The solution cannot reach the sphere $|y|=\varepsilon$, because every point on that sphere satisfies $L(y)\ge m_P\varepsilon^2$. Hence the trajectory remains inside $B(0,\varepsilon)\subset B(0,\varepsilon_*)$. Applying Gronwall's inequality to the displayed differential inequality gives \begin{align*} L(y(t)) \le L(y(0))\exp\left(-\frac{t}{2M_P}\right) \end{align*} for all times for which the solution remains in $B(0,r_0)$. The stability argument keeps it there. Since $\overline{B}(0,r_0)\subset V$ and $Y\in C^1(V;\mathbb{R}^n)$, the ODE continuation theorem prevents finite positive-time blow-up while the solution remains in this compact subset of $V$. Therefore the solution exists for all $t\ge 0$. Finally $m_P|y(t)|^2\le L(y(t))$, so $y(t)\to 0$ as $t\to\infty$. This proves asymptotic stability of $0$ for the translated system, and translation by $x=y+x^*$ gives asymptotic stability of $x^*$ for the original system.[/guided]

[step:Use the local center-stable trapping theorem to prove instability when an unstable eigenvalue exists] Assume that $A = JY_0$ has an eigenvalue $\lambda$ with $\operatorname{Re}(\lambda) > 0$. Let $E^u \subset \mathbb{R}^n$ denote the real generalized eigenspace corresponding to eigenvalues of $A$ with positive real part, and let $E^{cs} \subset \mathbb{R}^n$ denote the real generalized eigenspace corresponding to eigenvalues of $A$ with nonpositive real part. Then \begin{align*} \mathbb{R}^n = E^{cs} \oplus E^u \end{align*} and $E^u \ne \{0\}$. We use the following precise form of the local center-stable manifold theorem for autonomous $C^1$ vector fields. If $F: W \to \mathbb{R}^n$ is a $C^1$ vector field on an open neighbourhood $W$ of $0$, $F(0)=0$, and $JF_0$ admits the invariant real splitting $\mathbb{R}^n = E^{cs} \oplus E^u$ where the eigenvalues on $E^{cs}$ have nonpositive real part and the eigenvalues on $E^u$ have positive real part, then there are a radius $\rho_* > 0$ and an embedded $C^1$ submanifold $W^{cs}_{\mathrm{loc}} \subset B(0,\rho_*)$ such that $T_0W^{cs}_{\mathrm{loc}} = E^{cs}$ and every initial condition $y_0 \in B(0,\rho_*)$ whose forward solution remains in $B(0,\rho_*)$ for all $t \ge 0$ belongs to $W^{cs}_{\mathrm{loc}}$. This is the trapping form of the theorem: it allows center directions in $E^{cs}$ and requires only $C^1$ regularity of the vector field, producing a $C^1$ local center-stable manifold with the stated forward-trapping characterization. We verify the hypotheses of this theorem for $F=Y$. The set $V$ is an open neighbourhood of $0$, the translated vector field $Y: V \to \mathbb{R}^n$ is $C^1$, and $Y(0)=0$. Its Jacobian matrix at $0$ is $JY_0=A$. The real generalized eigenspaces $E^{cs}$ and $E^u$ are $A$-invariant, their direct sum is all of $\mathbb{R}^n$, the eigenvalues on $E^{cs}$ have nonpositive real part by definition, and the eigenvalues on $E^u$ have positive real part by definition. Hence the theorem gives a radius $\rho_* > 0$ and an embedded $C^1$ submanifold $W^{cs}_{\mathrm{loc}} \subset B(0,\rho_*)$ with the stated trapping property. Since $V$ is open and contains $0$, decrease the radius if necessary and choose $\rho \in (0,\rho_*]$ such that \begin{align*} \overline{B}(0,\rho) \subset V. \end{align*} Every solution that remains in $B(0,\rho)$ for all $t \ge 0$ also remains in $B(0,\rho_*)$, so the same trapping property applies at radius $\rho$. Because $E^u \ne \{0\}$, the subspace $E^{cs}$ is a proper subspace of $\mathbb{R}^n$. Hence the embedded submanifold $W^{cs}_{\mathrm{loc}}$ has positive codimension and contains no open neighbourhood of $0$ in $\mathbb{R}^n$. Therefore, for every $\delta > 0$, there exists $y_\delta \in B(0,\delta) \cap B(0,\rho)$ with $y_\delta \notin W^{cs}_{\mathrm{loc}}$. For such an initial point $y_\delta$, let $\gamma_\delta: [0,T_\delta) \to V$ denote the maximal forward solution of $\dot{y}=Y(y)$ with $\gamma_\delta(0)=y_\delta$. Suppose, for contradiction, that $|\gamma_\delta(t)| < \rho$ for every $t \in [0,T_\delta)$. Since $\overline{B}(0,\rho) \subset V$ is compact and $Y \in C^1(V;\mathbb{R}^n)$ is locally Lipschitz and bounded on a neighbourhood of $\overline{B}(0,\rho)$, the ODE continuation theorem prevents finite positive-time blow-up while the trajectory remains in $\overline{B}(0,\rho)$. Therefore $T_\delta=\infty$. The trapping property then implies $y_\delta \in W^{cs}_{\mathrm{loc}}$, contradicting the choice of $y_\delta$. Hence there exists $t_\delta \in [0,T_\delta)$ such that \begin{align*} |\gamma_\delta(t_\delta)| \ge \rho. \end{align*} Thus arbitrarily small initial data leave the fixed neighbourhood $B(0,\rho)$ of the equilibrium. This is exactly Lyapunov instability of $0$ for the translated system. Translating back by $x = y + x^*$ proves Lyapunov instability of $x^*$ for the original system. [/step]