[proofplan]
One direction follows directly from the lower central series: repeated adjoint action by a fixed element pushes vectors deeper into the lower central series, which eventually vanishes for a nilpotent Lie algebra. For the converse, we prove [Engel's theorem](/theorems/3798) in the special form needed here: if every adjoint map is nilpotent, then every nonzero quotient of $\mathfrak g$ has nonzero center. Applying this to successive quotients constructs the upper central series until it reaches all of $\mathfrak g$, which is equivalent to nilpotence.
[/proofplan]
[step:Show nilpotence forces every adjoint map to be nilpotent]
Assume that $\mathfrak g$ is nilpotent. Define the lower central series $(\gamma_m(\mathfrak g))_{m \geq 1}$ by
\begin{align*}
\gamma_1(\mathfrak g) &:= \mathfrak g, \\
\gamma_{m+1}(\mathfrak g) &:= [\mathfrak g,\gamma_m(\mathfrak g)] \qquad \text{for } m \geq 1.
\end{align*}
Since $\mathfrak g$ is nilpotent, there exists an integer $r \geq 1$ such that $\gamma_r(\mathfrak g)=0$.
Fix $x \in \mathfrak g$. We prove by induction on $m \geq 0$ that
\begin{align*}
(\operatorname{ad}x)^m(\mathfrak g) \subseteq \gamma_{m+1}(\mathfrak g).
\end{align*}
For $m=0$, this says $\mathfrak g \subseteq \gamma_1(\mathfrak g)$, which holds by definition. If the inclusion holds for $m$, then
\begin{align*}
(\operatorname{ad}x)^{m+1}(\mathfrak g)
&= [x,(\operatorname{ad}x)^m(\mathfrak g)] \\
&\subseteq [\mathfrak g,\gamma_{m+1}(\mathfrak g)] \\
&= \gamma_{m+2}(\mathfrak g).
\end{align*}
Thus the induction is complete. Taking $m=r-1$ gives
\begin{align*}
(\operatorname{ad}x)^{r-1}(\mathfrak g) \subseteq \gamma_r(\mathfrak g)=0.
\end{align*}
Hence $\operatorname{ad}x$ is nilpotent.
[guided]
Assume first that $\mathfrak g$ is nilpotent. The lower central series records all iterated commutators with elements of $\mathfrak g$. Define
\begin{align*}
\gamma_1(\mathfrak g) &:= \mathfrak g, \\
\gamma_{m+1}(\mathfrak g) &:= [\mathfrak g,\gamma_m(\mathfrak g)] \qquad \text{for } m \geq 1.
\end{align*}
Nilpotence means precisely that this descending sequence eventually reaches zero: there is an integer $r \geq 1$ with $\gamma_r(\mathfrak g)=0$.
Fix $x \in \mathfrak g$. The map $\operatorname{ad}x:\mathfrak g \to \mathfrak g$ is given by $y \mapsto [x,y]$. Applying it once to an element of $\mathfrak g$ produces an element of $[\mathfrak g,\mathfrak g]=\gamma_2(\mathfrak g)$. Applying it again produces a bracket of an element of $\mathfrak g$ with something already in $\gamma_2(\mathfrak g)$, hence lands in $\gamma_3(\mathfrak g)$. Formally, we prove by induction that
\begin{align*}
(\operatorname{ad}x)^m(\mathfrak g) \subseteq \gamma_{m+1}(\mathfrak g)
\end{align*}
for every integer $m \geq 0$.
For $m=0$, this is the identity inclusion $\mathfrak g \subseteq \gamma_1(\mathfrak g)$. Suppose it holds for $m$. Then
\begin{align*}
(\operatorname{ad}x)^{m+1}(\mathfrak g)
&= [x,(\operatorname{ad}x)^m(\mathfrak g)] \\
&\subseteq [\mathfrak g,\gamma_{m+1}(\mathfrak g)] \\
&= \gamma_{m+2}(\mathfrak g).
\end{align*}
The first equality is the definition of the adjoint action, the inclusion uses $x \in \mathfrak g$ together with the induction hypothesis, and the final equality is the recursive definition of the lower central series.
Now choose $m=r-1$. Since $\gamma_r(\mathfrak g)=0$, the inclusion gives
\begin{align*}
(\operatorname{ad}x)^{r-1}(\mathfrak g) \subseteq 0.
\end{align*}
Thus $(\operatorname{ad}x)^{r-1}=0$ as an endomorphism of $\mathfrak g$, so $\operatorname{ad}x$ is nilpotent.
[/guided]
[/step]
[step:Prove the Engel fixed vector lemma for nilpotent linear transformations]
[claim:Engel fixed vector lemma]
Let $V$ be a nonzero finite-dimensional [vector space](/page/Vector%20Space) over $F$, and let $\mathfrak l \subseteq \mathfrak{gl}(V)$ be a Lie subalgebra such that every $T \in \mathfrak l$ is nilpotent as an endomorphism of $V$. Then there exists $0 \neq v \in V$ such that $T(v)=0$ for every $T \in \mathfrak l$.
[/claim]
[proof]
We argue by induction on $d:=\dim_F \mathfrak l$. If $d=0$, then $\mathfrak l=0$, and every nonzero vector of $V$ has the required property.
Assume $d>0$ and that the result holds for all Lie algebras of nilpotent endomorphisms of dimension smaller than $d$. Choose a maximal proper Lie subalgebra $\mathfrak m \subsetneq \mathfrak l$. Such a subalgebra exists because $0$ is a proper Lie subalgebra when $d>0$ and $\mathfrak l$ is finite-dimensional. By the induction hypothesis applied to $\mathfrak m$, the subspace
\begin{align*}
W := \{v \in V : S(v)=0 \text{ for every } S \in \mathfrak m\}
\end{align*}
is nonzero.
We prove that $\mathfrak m$ is an ideal of codimension $1$ in $\mathfrak l$. Define its normalizer in $\mathfrak l$ by
\begin{align*}
N_{\mathfrak l}(\mathfrak m) := \{T \in \mathfrak l : [T,S] \in \mathfrak m \text{ for every } S \in \mathfrak m\}.
\end{align*}
It suffices to show $N_{\mathfrak l}(\mathfrak m) \neq \mathfrak m$, because maximality of $\mathfrak m$ then implies $N_{\mathfrak l}(\mathfrak m)=\mathfrak l$.
Let $Q:=\mathfrak l/\mathfrak m$ be the quotient vector space. For each $S \in \mathfrak m$, define the [linear map](/page/Linear%20Map)
\begin{align*}
\rho(S): Q &\to Q \\
T+\mathfrak m &\mapsto [S,T]+\mathfrak m.
\end{align*}
This is well-defined because $\mathfrak m$ is a Lie subalgebra: if $T-T' \in \mathfrak m$, then $[S,T-T'] \in \mathfrak m$. The assignment $S \mapsto \rho(S)$ is a Lie algebra homomorphism from $\mathfrak m$ to $\mathfrak{gl}(Q)$.
For each $S \in \mathfrak m$, the map $\rho(S)$ is nilpotent. Indeed, $S:V\to V$ is nilpotent by hypothesis. Let $L_S:\mathfrak{gl}(V)\to\mathfrak{gl}(V)$ and $R_S:\mathfrak{gl}(V)\to\mathfrak{gl}(V)$ be the left and right multiplication maps $L_S(A)=SA$ and $R_S(A)=AS$. If $S^q=0$, then $L_S^q=0$ and $R_S^q=0$, and $L_S$ commutes with $R_S$. Hence $L_S-R_S$ is nilpotent, since
\begin{align*}
(L_S-R_S)^{2q-1} = \sum_{j=0}^{2q-1} (-1)^j \binom{2q-1}{j} L_S^{2q-1-j}R_S^j =0;
\end{align*}
in every summand either $2q-1-j \geq q$ or $j \geq q$. The restriction of $L_S-R_S$ to $\mathfrak l$ is $\operatorname{ad}S:T\mapsto [S,T]$, and $\rho(S)$ is the induced map on the quotient $Q$. Therefore $\rho(S)$ is nilpotent.
If $Q$ were zero, then $\mathfrak m=\mathfrak l$, contrary to the choice of $\mathfrak m$; hence $Q\neq 0$. Since $\dim_F\mathfrak m<d$, the induction hypothesis applied to the Lie algebra $\rho(\mathfrak m)\subseteq\mathfrak{gl}(Q)$ gives a nonzero coset $T+\mathfrak m\in Q$ such that
\begin{align*}
\rho(S)(T+\mathfrak m)=0
\end{align*}
for every $S\in\mathfrak m$. This means $T\notin\mathfrak m$ and $[S,T]\in\mathfrak m$ for every $S\in\mathfrak m$. Since $[T,S]=-[S,T]$, we have $T\in N_{\mathfrak l}(\mathfrak m)\setminus\mathfrak m$. Thus $N_{\mathfrak l}(\mathfrak m)$ strictly contains $\mathfrak m$, so maximality gives $N_{\mathfrak l}(\mathfrak m)=\mathfrak l$. Therefore $\mathfrak m$ is an ideal of $\mathfrak l$.
The quotient Lie algebra $\mathfrak l/\mathfrak m$ has no nonzero proper Lie subalgebras, because the inverse image of any such subalgebra would be a Lie subalgebra strictly between $\mathfrak m$ and $\mathfrak l$. Since every one-dimensional subspace of a Lie algebra is a Lie subalgebra, it follows that $\dim_F(\mathfrak l/\mathfrak m)=1$. Choose $A\in\mathfrak l$ whose image spans $\mathfrak l/\mathfrak m$; then
\begin{align*}
\mathfrak l = \mathfrak m \oplus F A
\end{align*}
as vector spaces.
Because $\mathfrak m$ is an ideal, $W$ is stable under $A$. Indeed, if $v \in W$ and $S \in \mathfrak m$, then
\begin{align*}
S(A(v)) = A(S(v)) - [A,S](v) = 0 - 0 = 0,
\end{align*}
since $S(v)=0$ and $[A,S]\in\mathfrak m$. Thus $A(W)\subseteq W$.
The restriction $A|_W:W\to W$ is nilpotent because $A$ is nilpotent on $V$. Since $W\neq 0$, the kernel of $A|_W$ is nonzero. Choose $0\neq v\in W$ with $A(v)=0$. Then every element of $\mathfrak m$ kills $v$ by the definition of $W$, and $A$ kills $v$ by construction. Since $\mathfrak l=\mathfrak m\oplus F A$, every element of $\mathfrak l$ kills $v$.
[/proof]
[/step]
[step:Find a nonzero central element from nilpotent adjoint maps]
Assume now that $\operatorname{ad}x$ is nilpotent for every $x \in \mathfrak g$. If $\mathfrak g=0$, then $\mathfrak g$ is nilpotent. Suppose $\mathfrak g \neq 0$.
Apply the Engel fixed vector lemma with $V:=\mathfrak g$ and
\begin{align*}
\mathfrak l := \operatorname{ad}(\mathfrak g) = \{\operatorname{ad}x : x \in \mathfrak g\} \subseteq \mathfrak{gl}(\mathfrak g).
\end{align*}
This is a Lie subalgebra because
\begin{align*}
[\operatorname{ad}x,\operatorname{ad}y] = \operatorname{ad}[x,y]
\end{align*}
for all $x,y \in \mathfrak g$, by the Jacobi identity. By hypothesis, every element of $\mathfrak l$ is nilpotent. Hence there exists $0 \neq z \in \mathfrak g$ such that
\begin{align*}
(\operatorname{ad}x)(z)=0
\end{align*}
for every $x \in \mathfrak g$. Equivalently, $[x,z]=0$ for every $x \in \mathfrak g$, so $z \in Z(\mathfrak g)$. Therefore $Z(\mathfrak g) \neq 0$.
[guided]
Now assume that each adjoint map $\operatorname{ad}x$ is nilpotent. We want to prove that $\mathfrak g$ is nilpotent. The first structural consequence is that the center is nonzero.
If $\mathfrak g=0$, the conclusion is immediate, so assume $\mathfrak g \neq 0$. Consider the adjoint image
\begin{align*}
\mathfrak l := \operatorname{ad}(\mathfrak g) = \{\operatorname{ad}x : x \in \mathfrak g\} \subseteq \mathfrak{gl}(\mathfrak g).
\end{align*}
This is a Lie subalgebra of $\mathfrak{gl}(\mathfrak g)$. Indeed, for $x,y,u \in \mathfrak g$, the Jacobi identity gives
\begin{align*}
[\operatorname{ad}x,\operatorname{ad}y](u)
&= [x,[y,u]] - [y,[x,u]] \\
&= [[x,y],u] \\
&= (\operatorname{ad}[x,y])(u).
\end{align*}
Thus $[\operatorname{ad}x,\operatorname{ad}y]=\operatorname{ad}[x,y]$, so $\mathfrak l$ is closed under the commutator bracket.
The hypothesis says exactly that every element of $\mathfrak l$ is nilpotent as an endomorphism of the vector space $\mathfrak g$. The Engel fixed vector lemma therefore applies with $V=\mathfrak g$. It gives a nonzero element $z \in \mathfrak g$ such that
\begin{align*}
(\operatorname{ad}x)(z)=0
\end{align*}
for every $x \in \mathfrak g$. Since $(\operatorname{ad}x)(z)=[x,z]$, this means $[x,z]=0$ for every $x \in \mathfrak g$. Hence $z$ lies in the center
\begin{align*}
Z(\mathfrak g):=\{u \in \mathfrak g : [x,u]=0 \text{ for every } x \in \mathfrak g\}.
\end{align*}
Thus $Z(\mathfrak g)$ contains the nonzero element $z$.
[/guided]
[/step]
[step:Apply the central element argument to all nonzero quotients]
Let $\mathfrak a \trianglelefteq \mathfrak g$ be an ideal. Define the quotient Lie algebra $\mathfrak g/\mathfrak a$ with bracket
\begin{align*}
[x+\mathfrak a,y+\mathfrak a] := [x,y]+\mathfrak a.
\end{align*}
If $x \in \mathfrak g$, then the adjoint endomorphism of $\mathfrak g/\mathfrak a$ induced by $x+\mathfrak a$ satisfies
\begin{align*}
\operatorname{ad}_{\mathfrak g/\mathfrak a}(x+\mathfrak a)^m(y+\mathfrak a)
= (\operatorname{ad}_{\mathfrak g}x)^m(y)+\mathfrak a
\end{align*}
for every $m \geq 1$ and every $y \in \mathfrak g$. Since $\operatorname{ad}_{\mathfrak g}x$ is nilpotent, $\operatorname{ad}_{\mathfrak g/\mathfrak a}(x+\mathfrak a)$ is nilpotent. Therefore every nonzero quotient $\mathfrak g/\mathfrak a$ has nonzero center by the previous step.
[guided]
The previous step showed that a Lie algebra whose adjoint maps are all nilpotent has nonzero center. To build nilpotence of $\mathfrak g$, we need the same conclusion not only for $\mathfrak g$ itself but also for its quotients.
Let $\mathfrak a \trianglelefteq \mathfrak g$ be an ideal. The quotient vector space $\mathfrak g/\mathfrak a$ becomes a Lie algebra under
\begin{align*}
[x+\mathfrak a,y+\mathfrak a] := [x,y]+\mathfrak a.
\end{align*}
This bracket is well-defined because $\mathfrak a$ is an ideal. For $x \in \mathfrak g$, the adjoint map on the quotient is
\begin{align*}
\operatorname{ad}_{\mathfrak g/\mathfrak a}(x+\mathfrak a): \mathfrak g/\mathfrak a &\to \mathfrak g/\mathfrak a \\
y+\mathfrak a &\mapsto [x,y]+\mathfrak a.
\end{align*}
We compare this map with $\operatorname{ad}_{\mathfrak g}x$. A direct induction on $m \geq 1$ gives
\begin{align*}
\operatorname{ad}_{\mathfrak g/\mathfrak a}(x+\mathfrak a)^m(y+\mathfrak a)
= (\operatorname{ad}_{\mathfrak g}x)^m(y)+\mathfrak a.
\end{align*}
Because $\operatorname{ad}_{\mathfrak g}x$ is nilpotent by hypothesis, there exists an integer $m \geq 1$ such that $(\operatorname{ad}_{\mathfrak g}x)^m=0$. The displayed formula then gives
\begin{align*}
\operatorname{ad}_{\mathfrak g/\mathfrak a}(x+\mathfrak a)^m=0.
\end{align*}
Thus every adjoint map in the quotient is nilpotent. Applying the previous step to the nonzero Lie algebra $\mathfrak g/\mathfrak a$ shows that its center is nonzero.
[/guided]
[/step]
[step:Construct the upper central series until it reaches the whole algebra]
Define the upper central series $(Z_m(\mathfrak g))_{m \geq 0}$ by
\begin{align*}
Z_0(\mathfrak g) &:= 0, \\
Z_{m+1}(\mathfrak g)/Z_m(\mathfrak g) &:= Z(\mathfrak g/Z_m(\mathfrak g)).
\end{align*}
Each $Z_m(\mathfrak g)$ is an ideal of $\mathfrak g$. If $Z_m(\mathfrak g) \neq \mathfrak g$, then the quotient $\mathfrak g/Z_m(\mathfrak g)$ is nonzero, so its center is nonzero by the previous step. Hence
\begin{align*}
Z_m(\mathfrak g) \subsetneq Z_{m+1}(\mathfrak g).
\end{align*}
Because $\mathfrak g$ is finite-dimensional, this strictly increasing chain cannot continue for more than $\dim_F \mathfrak g$ steps. Therefore there exists $c \geq 0$ such that
\begin{align*}
Z_c(\mathfrak g)=\mathfrak g.
\end{align*}
[guided]
The upper central series grows by repeatedly adding elements that are central modulo what has already been added. Define
\begin{align*}
Z_0(\mathfrak g) &:= 0, \\
Z_{m+1}(\mathfrak g)/Z_m(\mathfrak g) &:= Z(\mathfrak g/Z_m(\mathfrak g)).
\end{align*}
Equivalently, $Z_{m+1}(\mathfrak g)$ is the inverse image of the center of $\mathfrak g/Z_m(\mathfrak g)$ under the quotient map $\mathfrak g \to \mathfrak g/Z_m(\mathfrak g)$. This description shows that each $Z_m(\mathfrak g)$ is an ideal.
Suppose $Z_m(\mathfrak g) \neq \mathfrak g$. Then the quotient $\mathfrak g/Z_m(\mathfrak g)$ is a nonzero finite-dimensional Lie algebra. By the quotient argument from the previous step, all its adjoint maps are nilpotent. Applying the nonzero-center result to this quotient gives
\begin{align*}
Z(\mathfrak g/Z_m(\mathfrak g)) \neq 0.
\end{align*}
By the definition of the upper central series, this means
\begin{align*}
Z_{m+1}(\mathfrak g)/Z_m(\mathfrak g) \neq 0,
\end{align*}
and therefore $Z_m(\mathfrak g) \subsetneq Z_{m+1}(\mathfrak g)$.
Since $\mathfrak g$ has finite dimension over $F$, a strictly increasing chain of subspaces of $\mathfrak g$ has length at most $\dim_F \mathfrak g$. Thus this process must stop, and the only possible stopping point is $\mathfrak g$ itself. Hence there exists an integer $c \geq 0$ such that
\begin{align*}
Z_c(\mathfrak g)=\mathfrak g.
\end{align*}
[/guided]
[/step]
[step:Conclude that the Lie algebra is nilpotent]
We show that $Z_c(\mathfrak g)=\mathfrak g$ implies nilpotence. For every $m \geq 0$, the defining property of the upper central series gives
\begin{align*}
[\mathfrak g,Z_{m+1}(\mathfrak g)] \subseteq Z_m(\mathfrak g).
\end{align*}
We prove by induction on $k \geq 0$ that
\begin{align*}
\gamma_{k+1}(\mathfrak g) \subseteq Z_{c-k}(\mathfrak g)
\end{align*}
for $0 \leq k \leq c$. For $k=0$, this is $\mathfrak g \subseteq Z_c(\mathfrak g)$, which holds because $Z_c(\mathfrak g)=\mathfrak g$. If the inclusion holds for $k$, then
\begin{align*}
\gamma_{k+2}(\mathfrak g)
&= [\mathfrak g,\gamma_{k+1}(\mathfrak g)] \\
&\subseteq [\mathfrak g,Z_{c-k}(\mathfrak g)] \\
&\subseteq Z_{c-k-1}(\mathfrak g).
\end{align*}
Thus $\gamma_{c+1}(\mathfrak g) \subseteq Z_0(\mathfrak g)=0$. Therefore the lower central series reaches zero, so $\mathfrak g$ is nilpotent.
Combining this with the first step proves the equivalence.
[/step]