[proofplan]
We pass to the reduced phase space at momentum $\mu$, where the relative equilibrium becomes an ordinary equilibrium. In local slice variables the reduced Hamiltonian is the sum of a positive kinetic energy in the reduced shape momenta and the amended potential in the reduced shape variables. The critical point hypothesis gives vanishing first variation, while the positive definiteness of $D^2V_\mu(q)$ and of the kinetic energy gives a strict local minimum of the reduced Hamiltonian. Conservation of reduced energy traps nearby reduced trajectories in an arbitrarily small reduced neighbourhood, and lifting this reduced stability statement gives Lyapunov stability modulo the momentum stabilizer $G_\mu$ for perturbations inside the fixed momentum level $J^{-1}(\mu)$.
[/proofplan]
[step:Pass from the relative equilibrium to an equilibrium on the reduced phase space]
Let
\begin{align*}
\mathbb{F}L:TQ\to T^*Q
\end{align*}
denote the Legendre transform of the simple mechanical Lagrangian, defined by the Riemannian metric on $Q$. Let
\begin{align*}
H:T^*Q\to\mathbb R
\end{align*}
denote the Hamiltonian obtained from $L$ through this Legendre transform, and let
\begin{align*}
J:T^*Q\to\mathfrak g^*
\end{align*}
denote the momentum map for the lifted cotangent action. Define the momentum stabilizer
\begin{align*}
G_\mu:=\{g\in G:\operatorname{Ad}^*_g\mu=\mu\}.
\end{align*}
Let $P_\mu$ denote the local reduced phase space near the reduced point:
\begin{align*}
P_\mu:=J^{-1}(\mu)/G_\mu.
\end{align*}
Since $\mu$ is a regular value and $P_\mu$ is smooth near the point under consideration, $P_\mu$ is a finite-dimensional smooth manifold locally. Let $z_\mu\in P_\mu$ denote the image of the relative equilibrium with representative $q\in Q$.
The Hamiltonian $H$ on $T^*Q$ induced by the simple mechanical Lagrangian is $G$-invariant because the action is isometric and $V$ is $G$-invariant. Hence it descends locally to a smooth reduced Hamiltonian
\begin{align*}
h_\mu:P_\mu\to\mathbb{R}.
\end{align*}
A relative equilibrium at momentum $\mu$ is precisely an equilibrium of the reduced Hamiltonian vector field on $P_\mu$. Thus it is enough to prove Lyapunov stability of $z_\mu$ for the reduced dynamics.
[guided]
The point of reduction is that a relative equilibrium is not usually a fixed point in the original phase space: its trajectory may move along the symmetry group. After fixing the momentum value $\mu$ and quotienting by the momentum stabilizer $G_\mu$, that group motion is removed.
Let $g$ denote the Riemannian metric on $Q$, so for each $q\in Q$ the symbol $g_q:T_qQ\times T_qQ\to\mathbb R$ denotes the [inner product](/page/Inner%20Product) on the tangent space at $q$. The simple mechanical Lagrangian has the form
\begin{align*}
L:TQ\to\mathbb R, \qquad L(v_q)=\frac{1}{2}g_q(v_q,v_q)-V(q).
\end{align*}
Let
\begin{align*}
\mathbb{F}L:TQ\to T^*Q
\end{align*}
be the Legendre transform induced by $g$, and let
\begin{align*}
H:T^*Q\to\mathbb R
\end{align*}
be the Hamiltonian obtained from $L$ through $\mathbb{F}L$. Let
\begin{align*}
J:T^*Q\to\mathfrak g^*
\end{align*}
be the momentum map for the lifted cotangent action, and define
\begin{align*}
G_\mu:=\{g\in G:\operatorname{Ad}^*_g\mu=\mu\}.
\end{align*}
Then the local reduced phase space is
\begin{align*}
P_\mu:=J^{-1}(\mu)/G_\mu.
\end{align*}
By hypothesis, $P_\mu$ is a smooth finite-dimensional manifold in this neighbourhood. Let $z_\mu\in P_\mu$ be the point represented by the original relative equilibrium.
The Hamiltonian $H$ is invariant under the lifted cotangent action: the kinetic term is invariant because $G$ acts by isometries on $(Q,g)$, and the potential term is invariant because $V$ is $G$-invariant. Therefore the Hamiltonian restricts to $J^{-1}(\mu)$ and descends to a smooth function
\begin{align*}
h_\mu:P_\mu\to\mathbb{R}.
\end{align*}
The reduced Hamiltonian vector field has $z_\mu$ as an equilibrium exactly because the unreduced trajectory differs from a fixed point only by motion along the symmetry group. Thus proving Lyapunov stability of $z_\mu$ in $P_\mu$ proves fixed-momentum stability of the original trajectory modulo the $G_\mu$-action.
[/guided]
[/step]
[step:Write the reduced Hamiltonian in slice coordinates]
By the local normal-form hypothesis in the statement, there is a chart
\begin{align*}
\Phi:U_\mu\to \Phi(U_\mu)\subset W_q\times W_q^*
\end{align*}
from a neighbourhood $U_\mu\subset P_\mu$ of $z_\mu$ such that $\Phi(z_\mu)=(0,0)$. Write $\Phi(y)=(x,\pi)$, where $x\in W_q$ is the reduced shape displacement and $\pi\in W_q^*$ is the corresponding reduced shape momentum. In these coordinates the amended potential is a smooth map
\begin{align*}
V_\mu:S\to\mathbb R,
\end{align*}
viewed in the slice coordinate $x$ near $0$, and the reduced Hamiltonian has the form
\begin{align*}
h_\mu(x,\pi)=K_\mu(x,\pi)+V_\mu(x),
\end{align*}
where
\begin{align*}
K_\mu:W_q\times W_q^*\to\mathbb{R}
\end{align*}
is smooth and, for each $x$ near $0$, the map
\begin{align*}
W_q^*\to\mathbb R, \qquad \pi\mapsto K_\mu(x,\pi)
\end{align*}
is a positive definite quadratic form on $W_q^*$ satisfying $D_\pi K_\mu(x,0)=0$. The reduced equilibrium corresponding to $m$ is represented by $(0,0)$, and the hypothesis that $q$ is a critical point of $V_\mu$ restricted to the slice gives
\begin{align*}
Dh_\mu(0,0)=0.
\end{align*}
[guided]
The theorem is stated with the exact local reduced normal form needed for the energy argument. Thus no global claim about all possible nonabelian reduced spaces is being made here. In a neighbourhood of $z_\mu$, the reduced Hamiltonian has local coordinates $(x,\pi)\in W_q\times W_q^*$ centered at $z_\mu$ in which it separates as amended potential plus reduced kinetic energy:
\begin{align*}
h_\mu(x,\pi)=K_\mu(x,\pi)+V_\mu(x).
\end{align*}
More precisely, the kinetic term satisfies $K_\mu(x,0)=0$, $D_\pi K_\mu(x,0)=0$, and $\pi\mapsto K_\mu(x,\pi)$ is a positive definite quadratic form for each $x$ near $0$. These identities are exactly what remove linear momentum terms and give the positive momentum Hessian block used below.
Let
\begin{align*}
W_q:=T_qS.
\end{align*}
The normal form gives a chart
\begin{align*}
\Phi:U_\mu\to \Phi(U_\mu)\subset W_q\times W_q^*
\end{align*}
from a neighbourhood $U_\mu\subset P_\mu$ of $z_\mu$ such that $\Phi(z_\mu)=(0,0)$. If $y\in U_\mu$, write $\Phi(y)=(x,\pi)$, where $x\in W_q$ is the reduced shape displacement and $\pi\in W_q^*$ is the reduced shape momentum.
The amended potential is a smooth map
\begin{align*}
V_\mu:S\to\mathbb R,
\end{align*}
viewed in the local shape coordinate $x$ near $0$. The normal form identifies the reduced Hamiltonian as
\begin{align*}
h_\mu(x,\pi)=K_\mu(x,\pi)+V_\mu(x),
\end{align*}
where
\begin{align*}
K_\mu:W_q\times W_q^*\to\mathbb R
\end{align*}
is smooth and, for each $x$ near $0$, the map
\begin{align*}
W_q^*\to\mathbb R, \qquad \pi\mapsto K_\mu(x,\pi)
\end{align*}
is a positive definite quadratic form. This precise form is what excludes extra linear momentum terms at the equilibrium and is the reason the later Hessian computation separates into a shape block and a momentum block.
The reduced equilibrium is represented by $(0,0)$. Since the theorem assumes that the relative equilibrium corresponds locally to a critical point of $V_\mu$ on the slice, and since the kinetic energy has no linear momentum term at $(0,0)$, the first derivative of the reduced Hamiltonian vanishes:
\begin{align*}
Dh_\mu(0,0)=0.
\end{align*}
[/guided]
[/step]
[step:Show that the second variation of the reduced Hamiltonian is positive definite]
Let $(\xi,\alpha)\in W_q\times W_q^*$. Since the kinetic term is quadratic in the momentum variable and has no linear momentum part at $\pi=0$, the second variation at $(0,0)$ splits as
\begin{align*}
D^2h_\mu(0,0)[(\xi,\alpha),(\xi,\alpha)] = D^2V_\mu(q)[\xi,\xi]+D^2_{\pi\pi}K_\mu(0,0)[\alpha,\alpha].
\end{align*}
By hypothesis, $D^2V_\mu(q)$ is positive definite on $W_q$. By positive definiteness of the reduced kinetic energy, $D^2_{\pi\pi}K_\mu(0,0)$ is positive definite on $W_q^*$. Therefore
\begin{align*}
D^2h_\mu(0,0)[(\xi,\alpha),(\xi,\alpha)]>0
\end{align*}
for every nonzero $(\xi,\alpha)\in W_q\times W_q^*$.
[guided]
We now check why the energy has a positive quadratic approximation in every reduced direction. A tangent vector to the reduced phase space in these local coordinates has the form
\begin{align*}
(\xi,\alpha)\in W_q\times W_q^*,
\end{align*}
where $\xi$ is a shape direction and $\alpha$ is a reduced momentum direction.
The reduced Hamiltonian is
\begin{align*}
h_\mu(x,\pi)=K_\mu(x,\pi)+V_\mu(x).
\end{align*}
At the equilibrium, the reduced shape momentum is $\pi=0$. For each fixed $x$ near $0$, the function $\pi\mapsto K_\mu(x,\pi)$ is a quadratic form, so $K_\mu(x,0)=0$ and $D_\pi K_\mu(x,0)=0$. Differentiating these identities in the $x$-directions at $x=0$ shows that the pure shape and mixed shape-momentum second derivatives of $K_\mu$ vanish at $(0,0)$. Also, $V_\mu$ depends only on the shape variable. Hence the quadratic part of $h_\mu$ at $(0,0)$ separates into the shape contribution and the momentum contribution:
\begin{align*}
D^2h_\mu(0,0)[(\xi,\alpha),(\xi,\alpha)] = D^2V_\mu(q)[\xi,\xi]+D^2_{\pi\pi}K_\mu(0,0)[\alpha,\alpha].
\end{align*}
The first term is positive for every nonzero $\xi\in W_q$ by the hypothesis that $D^2V_\mu(q)$ is positive definite on the reduced shape tangent space. The second term is positive for every nonzero $\alpha\in W_q^*$ because it is the quadratic form associated with the reduced kinetic energy. Therefore the sum is positive whenever $(\xi,\alpha)\ne(0,0)$. This proves that the Hessian of the reduced Hamiltonian is positive definite at the reduced equilibrium.
[/guided]
[/step]
[step:Convert positive definiteness into a strict local energy minimum]
Choose any Euclidean norm $|\cdot|$ on the finite-dimensional [vector space](/page/Vector%20Space) $W_q\times W_q^*$. Since $D^2h_\mu(0,0)$ is positive definite, compactness of the unit sphere gives a number $m>0$ such that
\begin{align*}
D^2h_\mu(0,0)[\eta,\eta]\ge m|\eta|^2
\end{align*}
for every $\eta\in W_q\times W_q^*$. By continuity of the Hessian, there is $r>0$ such that
\begin{align*}
D^2h_\mu(\theta\eta)[\eta,\eta]\ge \frac{m}{2}|\eta|^2
\end{align*}
whenever $|\eta|<r$ and $0\le \theta\le 1$. [Taylor's theorem with integral remainder](/theorems/189), applied to $h_\mu$ along the line segment $\theta\mapsto \theta\eta$, and the identity $Dh_\mu(0,0)=0$ give
\begin{align*}
h_\mu(\eta)\ge h_\mu(0,0)+\frac{m}{4}|\eta|^2
\end{align*}
for $|\eta|<r$. Thus $(0,0)$ is a strict local minimum of $h_\mu$. Equivalently, $z_\mu$ is a strict local minimum of the reduced Hamiltonian on $P_\mu$.
[/step]
[step:Use conservation of reduced energy to trap nearby reduced trajectories]
Let $\mathcal{O}_\mu$ be any neighbourhood of $z_\mu$ in $P_\mu$. Choose the slice chart $\Phi:U_\mu\to W_q\times W_q^*$ from the previous step and choose $\rho>0$ so small that the closed coordinate ball
\begin{align*}
\overline{B}_\rho:=\{\eta\in W_q\times W_q^*:|\eta|\le \rho\}
\end{align*}
is contained in $\Phi(U_\mu)$ and $\Phi^{-1}(\overline{B}_\rho)\subset\mathcal{O}_\mu$. Since $z_\mu$ is a strict local minimum, after decreasing $\rho$ if necessary, the boundary sphere
\begin{align*}
\partial B_\rho:=\{\eta\in W_q\times W_q^*:|\eta|=\rho\}
\end{align*}
satisfies
\begin{align*}
\delta:=\min_{\eta\in\partial B_\rho}\bigl(h_\mu(\Phi^{-1}(\eta))-h_\mu(z_\mu)\bigr)>0.
\end{align*}
The minimum exists because $\partial B_\rho$ is compact and the displayed function is continuous; it is positive because the strict local minimum excludes equality on the boundary.
By continuity of $h_\mu$, there exists a neighbourhood $\mathcal{V}_\mu\subset\Phi^{-1}(B_\rho)$ of $z_\mu$ such that
\begin{align*}
h_\mu(y)<h_\mu(z_\mu)+\delta
\end{align*}
for every $y\in\mathcal{V}_\mu$. Let $y:[0,T)\to P_\mu$ be a reduced trajectory with $y(0)\in\mathcal{V}_\mu$. Conservation of the reduced Hamiltonian gives
\begin{align*}
h_\mu(y(t))=h_\mu(y(0))<h_\mu(z_\mu)+\delta
\end{align*}
for every $t\in[0,T)$.
If $y(t)$ left $\Phi^{-1}(B_\rho)$ at some time, continuity of $t\mapsto\Phi(y(t))$ would give a first time $t_*$ with $\Phi(y(t_*))\in\partial B_\rho$. At that time the definition of $\delta$ would imply
\begin{align*}
h_\mu(y(t_*))\ge h_\mu(z_\mu)+\delta,
\end{align*}
contradicting the conserved-energy inequality. Hence $y(t)\in\Phi^{-1}(B_\rho)\subset\mathcal{O}_\mu$ for all $t\in[0,T)$. Therefore $z_\mu$ is Lyapunov stable for the reduced dynamics.
[guided]
The subtle point is that an energy inequality alone does not trap the trajectory unless the sublevel set is chosen around the correct connected local well. We build that local well using a compact boundary argument.
Let $\mathcal{O}_\mu$ be a neighbourhood of $z_\mu$ in $P_\mu$. In the slice chart $\Phi:U_\mu\to W_q\times W_q^*$, choose $\rho>0$ such that
\begin{align*}
\overline{B}_\rho:=\{\eta\in W_q\times W_q^*:|\eta|\le \rho\}
\end{align*}
is contained in $\Phi(U_\mu)$ and $\Phi^{-1}(\overline{B}_\rho)\subset\mathcal{O}_\mu$. The boundary sphere is
\begin{align*}
\partial B_\rho:=\{\eta\in W_q\times W_q^*:|\eta|=\rho\}.
\end{align*}
Because $z_\mu$ is a strict local minimum, we may decrease $\rho$ so that no boundary point has the same energy as $z_\mu$. The [continuous function](/page/Continuous%20Function) $\eta\mapsto h_\mu(\Phi^{-1}(\eta))-h_\mu(z_\mu)$ therefore has a positive minimum on the compact set $\partial B_\rho$:
\begin{align*}
\delta:=\min_{\eta\in\partial B_\rho}\bigl(h_\mu(\Phi^{-1}(\eta))-h_\mu(z_\mu)\bigr)>0.
\end{align*}
This number is the energy barrier that a trajectory must cross before it can leave the coordinate ball.
By continuity of $h_\mu$, choose a neighbourhood $\mathcal{V}_\mu\subset\Phi^{-1}(B_\rho)$ of $z_\mu$ such that
\begin{align*}
h_\mu(y)<h_\mu(z_\mu)+\delta
\end{align*}
for every $y\in\mathcal{V}_\mu$. Let $y:[0,T)\to P_\mu$ be a reduced trajectory with $y(0)\in\mathcal{V}_\mu$. Since $y$ is a Hamiltonian trajectory for $h_\mu$, the Hamiltonian is conserved along $y$:
\begin{align*}
h_\mu(y(t))=h_\mu(y(0))<h_\mu(z_\mu)+\delta
\end{align*}
for every $t\in[0,T)$.
Suppose, for contradiction, that the trajectory leaves $\Phi^{-1}(B_\rho)$. Since $t\mapsto\Phi(y(t))$ is continuous and starts inside $B_\rho$, there is a first exit time $t_*$ with $\Phi(y(t_*))\in\partial B_\rho$. By the definition of $\delta$,
\begin{align*}
h_\mu(y(t_*))\ge h_\mu(z_\mu)+\delta.
\end{align*}
This contradicts [conservation of energy](/theorems/1335) and the initial choice of $\mathcal{V}_\mu$. Therefore the trajectory remains in $\Phi^{-1}(B_\rho)\subset\mathcal{O}_\mu$ for all forward times for which it is defined. This is exactly Lyapunov stability of $z_\mu$ in the reduced phase space.
[/guided]
[/step]
[step:Lift reduced Lyapunov stability to fixed-momentum stability modulo $G_\mu$]
Let $\mathcal{U}$ be a $G_\mu$-invariant neighbourhood, relative to $J^{-1}(\mu)$, of the original $G_\mu$-orbit of the relative equilibrium in the fixed momentum level. Let
\begin{align*}
\rho_\mu:J^{-1}(\mu)\to P_\mu=J^{-1}(\mu)/G_\mu
\end{align*}
denote the quotient projection. Since $\rho_\mu$ is the quotient map for the $G_\mu$-action, it is an open map onto $P_\mu$. Because $\mathcal{U}$ is $G_\mu$-invariant, it is saturated for $\rho_\mu$, and therefore
\begin{align*}
\rho_\mu^{-1}(\rho_\mu(\mathcal{U}))=\mathcal{U}.
\end{align*}
The set
\begin{align*}
\mathcal{O}_\mu:=\rho_\mu(\mathcal{U})
\end{align*}
is consequently an open neighbourhood of $z_\mu$ in $P_\mu$, and it satisfies
\begin{align*}
\rho_\mu^{-1}(\mathcal{O}_\mu)\subset\mathcal{U}.
\end{align*}
By reduced Lyapunov stability, there is a neighbourhood $\mathcal{V}_\mu$ of $z_\mu$ whose reduced forward trajectories remain in $\mathcal{O}_\mu$. Define the fixed-momentum neighbourhood
\begin{align*}
\mathcal{V}:=\rho_\mu^{-1}(\mathcal{V}_\mu)\subset J^{-1}(\mu).
\end{align*}
If an unreduced trajectory starts in $\mathcal{V}$ and has momentum $\mu$, momentum conservation keeps it in $J^{-1}(\mu)$, and its projection under $\rho_\mu$ is the corresponding reduced trajectory. Hence its projected trajectory remains in $\mathcal{O}_\mu$, so the unreduced trajectory remains in $\rho_\mu^{-1}(\mathcal{O}_\mu)\subset\mathcal{U}$. This proves Lyapunov stability modulo $G_\mu$ for perturbations in the fixed momentum level $J^{-1}(\mu)$.
[guided]
The lift must be stated at fixed momentum. The reduced space $P_\mu$ only sees initial conditions in $J^{-1}(\mu)$, so the argument cannot control perturbations with different momentum values without additional hypotheses.
Let $\mathcal{U}$ be a $G_\mu$-invariant neighbourhood of the original $G_\mu$-orbit of the relative equilibrium, taken relative to the fixed momentum level $J^{-1}(\mu)$. Define the quotient projection
\begin{align*}
\rho_\mu:J^{-1}(\mu)\to P_\mu=J^{-1}(\mu)/G_\mu.
\end{align*}
The neighbourhood $\mathcal{U}$ is $G_\mu$-invariant, so it is saturated with respect to the quotient projection: if one point of a $G_\mu$-orbit lies in $\mathcal{U}$, the whole orbit lies in $\mathcal{U}$. Hence
\begin{align*}
\rho_\mu^{-1}(\rho_\mu(\mathcal{U}))=\mathcal{U}.
\end{align*}
Also, $\rho_\mu$ is an open quotient map, so the image
\begin{align*}
\mathcal{O}_\mu:=\rho_\mu(\mathcal{U})
\end{align*}
is an open neighbourhood of $z_\mu$ in $P_\mu$. This choice gives
\begin{align*}
\rho_\mu^{-1}(\mathcal{O}_\mu)\subset\mathcal{U}.
\end{align*}
Reduced Lyapunov stability supplies a neighbourhood $\mathcal{V}_\mu$ of $z_\mu$ such that every reduced trajectory beginning in $\mathcal{V}_\mu$ remains in $\mathcal{O}_\mu$ for all forward times for which it is defined. Set
\begin{align*}
\mathcal{V}:=\rho_\mu^{-1}(\mathcal{V}_\mu)\subset J^{-1}(\mu).
\end{align*}
Now take any unreduced trajectory with initial condition in $\mathcal{V}$. Since the Hamiltonian is $G$-invariant, the momentum map is conserved along the Hamiltonian flow; because the initial condition lies in $J^{-1}(\mu)$, the whole trajectory stays in $J^{-1}(\mu)$. Its image under $\rho_\mu$ is therefore a reduced trajectory beginning in $\mathcal{V}_\mu$, so it remains in $\mathcal{O}_\mu$. Pulling this containment back gives that the unreduced trajectory remains in $\rho_\mu^{-1}(\mathcal{O}_\mu)\subset\mathcal{U}$.
Thus the original relative equilibrium is Lyapunov stable modulo the $G_\mu$-action for perturbations lying in the fixed momentum level $J^{-1}(\mu)$. This is the exact lift of reduced Lyapunov stability from $P_\mu=J^{-1}(\mu)/G_\mu$.
[/guided]
[/step]