[proofplan] We first prove the invariant vector lemma: a finite-dimensional Lie algebra of nilpotent linear endomorphisms has a common nonzero vector killed by all its elements. Applying this lemma to the adjoint Lie algebra $\operatorname{ad}_{\mathfrak g}(\mathfrak g) \subseteq \operatorname{End}_F(\mathfrak g)$ gives a nonzero element of the centre $Z(\mathfrak g)$. We then pass to the quotient $\mathfrak g/Z(\mathfrak g)$, observe that the induced adjoint endomorphisms are still nilpotent, and use induction on $\dim_F \mathfrak g$. Finally, we lift nilpotence from the quotient by using the lower central series. [/proofplan]

[step:Prove the invariant vector lemma for nilpotent Lie algebras of endomorphisms][claim:Invariant vector lemma] Let $V$ be a nonzero finite-dimensional [vector space](/page/Vector%20Space) over $F$, and let $\mathfrak l \subseteq \operatorname{End}_F(V)$ be a finite-dimensional Lie subalgebra under the commutator bracket \begin{align*} [S,T]_{\operatorname{End}} := S \circ T - T \circ S. \end{align*} Assume that every $T \in \mathfrak l$ is nilpotent as an endomorphism of $V$. Then there exists $0 \ne v \in V$ such that $T(v)=0$ for every $T \in \mathfrak l$. [/claim] [proof] We prove the claim by induction on $m := \dim_F \mathfrak l$. If $m=0$, then every nonzero $v \in V$ is killed by every element of $\mathfrak l$. Suppose $m>0$, and assume the result for all Lie algebras of nilpotent endomorphisms of dimension strictly smaller than $m$. If $m=1$, choose $T_0 \in \mathfrak l$ with $\mathfrak l = F T_0$. Since $T_0$ is nilpotent and $V \ne 0$, the kernel $\ker T_0$ is nonzero. Any $0 \ne v \in \ker T_0$ is killed by every element of $\mathfrak l$. Now assume $m>1$. Choose a maximal proper Lie subalgebra $\mathfrak m \subsetneq \mathfrak l$, which exists because $\mathfrak l$ is finite-dimensional and has proper subalgebras, for instance $F T$ for any nonzero $T \in \mathfrak l$ if $m>1$. Define the normalizer of $\mathfrak m$ in $\mathfrak l$ by \begin{align*} N_{\mathfrak l}(\mathfrak m) := \{A \in \mathfrak l : [A,M]_{\operatorname{End}} \in \mathfrak m \text{ for every } M \in \mathfrak m\}. \end{align*} We first show that $N_{\mathfrak l}(\mathfrak m)$ strictly contains $\mathfrak m$. For each $M \in \mathfrak m$, define the induced [linear map](/page/Linear%20Map) \begin{align*} \rho(M): \mathfrak l/\mathfrak m &\to \mathfrak l/\mathfrak m \\ A+\mathfrak m &\mapsto [M,A]_{\operatorname{End}}+\mathfrak m. \end{align*} This map is well-defined because if $A-A' \in \mathfrak m$, then $[M,A-A']_{\operatorname{End}} \in \mathfrak m$ since $\mathfrak m$ is a Lie subalgebra. The assignment \begin{align*} \rho:\mathfrak m &\to \operatorname{End}_F(\mathfrak l/\mathfrak m) \\ M &\mapsto \rho(M) \end{align*} is a Lie algebra homomorphism. For each $M \in \mathfrak m$, the endomorphism $\rho(M)$ is nilpotent. Indeed, $M$ is nilpotent on $V$. Let \begin{align*} L_M:\operatorname{End}_F(V)&\to \operatorname{End}_F(V), & A&\mapsto M\circ A,\\ R_M:\operatorname{End}_F(V)&\to \operatorname{End}_F(V), & A&\mapsto A\circ M \end{align*} denote left and right multiplication by $M$. If $M^r=0$, then $L_M^r=0$ and $R_M^r=0$. Moreover $L_M R_M = R_M L_M$. Hence \begin{align*} \operatorname{ad}_{\operatorname{End}}(M)=L_M-R_M \end{align*} is nilpotent, because a polynomial in two commuting nilpotent endomorphisms with no constant term is nilpotent. Since $\rho(M)$ is the endomorphism induced by $\operatorname{ad}_{\operatorname{End}}(M)$ on the quotient $\mathfrak l/\mathfrak m$, it is nilpotent. The induction hypothesis applied to the Lie algebra $\rho(\mathfrak m)\subseteq \operatorname{End}_F(\mathfrak l/\mathfrak m)$ gives a nonzero coset $A+\mathfrak m \in \mathfrak l/\mathfrak m$ such that \begin{align*} \rho(M)(A+\mathfrak m)=0 \end{align*} for every $M \in \mathfrak m$. Thus $A \notin \mathfrak m$ and $[M,A]_{\operatorname{End}}\in \mathfrak m$ for every $M \in \mathfrak m$. Since $[A,M]_{\operatorname{End}}=-[M,A]_{\operatorname{End}}$, we have $A \in N_{\mathfrak l}(\mathfrak m)$. Therefore $N_{\mathfrak l}(\mathfrak m)$ strictly contains $\mathfrak m$. By maximality of $\mathfrak m$, the Lie subalgebra $N_{\mathfrak l}(\mathfrak m)$ must be all of $\mathfrak l$. Hence $\mathfrak m$ is an ideal of $\mathfrak l$. Since $\mathfrak m$ is maximal proper and $\mathfrak l/\mathfrak m$ has no nonzero proper Lie subalgebras, $\dim_F(\mathfrak l/\mathfrak m)=1$. By induction applied to $\mathfrak m \subseteq \operatorname{End}_F(V)$, the subspace \begin{align*} W := \{v \in V : M(v)=0 \text{ for every } M \in \mathfrak m\} \end{align*} is nonzero. The subspace $W$ is invariant under every $A \in \mathfrak l$. Indeed, if $v \in W$, $M \in \mathfrak m$, and $A \in \mathfrak l$, then \begin{align*} M(A(v)) = A(M(v)) + [M,A]_{\operatorname{End}}(v)=0+0=0, \end{align*} because $M(v)=0$ and $[M,A]_{\operatorname{End}}\in \mathfrak m$. Choose $A_0 \in \mathfrak l$ such that $\mathfrak l=\mathfrak m \oplus F A_0$ as vector spaces. Since $A_0$ is nilpotent on $V$ and $W$ is $A_0$-invariant, the restricted endomorphism \begin{align*} A_0|_W: W &\to W \end{align*} is nilpotent. Hence $\ker(A_0|_W)$ is nonzero. Choose $0 \ne v \in \ker(A_0|_W)$. Then $M(v)=0$ for every $M \in \mathfrak m$, and $A_0(v)=0$. Since every element of $\mathfrak l$ has the form $M+\lambda A_0$ with $M \in \mathfrak m$ and $\lambda \in F$, every element of $\mathfrak l$ kills $v$. [/proof][/step]

[guided]We prove a common-kernel statement for a whole Lie algebra of nilpotent endomorphisms. The proof is by induction on the dimension of the Lie algebra $\mathfrak l$, not on the dimension of the vector space $V$. The case $\dim_F \mathfrak l=0$ is immediate from the definition: there are no nonzero operators to impose conditions. If $\dim_F \mathfrak l=1$, then $\mathfrak l=F T_0$ for some nilpotent endomorphism $T_0:V\to V$. Since $V\ne 0$ and $T_0$ is nilpotent, $T_0$ cannot be injective unless $V=0$. Thus $\ker T_0\ne 0$, and any nonzero vector in $\ker T_0$ is killed by all of $\mathfrak l$. Now suppose $\dim_F\mathfrak l>1$. Choose a maximal proper Lie subalgebra $\mathfrak m\subsetneq \mathfrak l$. We want to prove that $\mathfrak m$ is almost all of $\mathfrak l$, specifically an ideal of codimension one. The device that detects this is the normalizer \begin{align*} N_{\mathfrak l}(\mathfrak m) := \{A \in \mathfrak l : [A,M]_{\operatorname{End}} \in \mathfrak m \text{ for every } M \in \mathfrak m\}. \end{align*} If we can find an element of $N_{\mathfrak l}(\mathfrak m)$ outside $\mathfrak m$, maximality will force $N_{\mathfrak l}(\mathfrak m)=\mathfrak l$, and then $\mathfrak m$ will be an ideal. To find such an element, let $\mathfrak m$ act on the quotient vector space $\mathfrak l/\mathfrak m$. For each $M\in\mathfrak m$, define \begin{align*} \rho(M): \mathfrak l/\mathfrak m &\to \mathfrak l/\mathfrak m \\ A+\mathfrak m &\mapsto [M,A]_{\operatorname{End}}+\mathfrak m. \end{align*} This is well-defined because replacing $A$ by $A+M_1$ with $M_1\in\mathfrak m$ changes $[M,A]_{\operatorname{End}}$ by $[M,M_1]_{\operatorname{End}}$, which lies in $\mathfrak m$ since $\mathfrak m$ is a Lie subalgebra. The important point is that each $\rho(M)$ is nilpotent. Since $M$ is nilpotent on $V$, choose $r\in\mathbb N$ with $M^r=0$. Define left and right multiplication operators on $\operatorname{End}_F(V)$ by \begin{align*} L_M:\operatorname{End}_F(V)&\to \operatorname{End}_F(V), & A&\mapsto M\circ A,\\ R_M:\operatorname{End}_F(V)&\to \operatorname{End}_F(V), & A&\mapsto A\circ M. \end{align*} Then $L_M^r=0$ and $R_M^r=0$, and $L_M$ commutes with $R_M$. The commutator action of $M$ on $\operatorname{End}_F(V)$ is \begin{align*} \operatorname{ad}_{\operatorname{End}}(M)=L_M-R_M. \end{align*} Because $L_M$ and $R_M$ commute and are nilpotent, $L_M-R_M$ is nilpotent. The map $\rho(M)$ is induced by this nilpotent map on the quotient $\mathfrak l/\mathfrak m$, so $\rho(M)$ is nilpotent. The induction hypothesis now applies to the Lie algebra $\rho(\mathfrak m)\subseteq \operatorname{End}_F(\mathfrak l/\mathfrak m)$. Therefore there is a nonzero coset $A+\mathfrak m$ killed by every $\rho(M)$. This means \begin{align*} [M,A]_{\operatorname{End}}\in\mathfrak m \end{align*} for every $M\in\mathfrak m$. Since $A+\mathfrak m\ne 0$, we have $A\notin\mathfrak m$, and since $[A,M]_{\operatorname{End}}=-[M,A]_{\operatorname{End}}$, we have $A\in N_{\mathfrak l}(\mathfrak m)$. Thus the normalizer strictly contains $\mathfrak m$. By maximality, $N_{\mathfrak l}(\mathfrak m)=\mathfrak l$. Hence $[A,M]_{\operatorname{End}}\in\mathfrak m$ for all $A\in\mathfrak l$ and $M\in\mathfrak m$, so $\mathfrak m$ is an ideal. The quotient $\mathfrak l/\mathfrak m$ has no nonzero proper Lie subalgebras, because any such subalgebra would lift to a Lie subalgebra strictly between $\mathfrak m$ and $\mathfrak l$. Therefore $\dim_F(\mathfrak l/\mathfrak m)=1$, since any vector space of dimension at least two has a one-dimensional proper Lie subalgebra spanned by a nonzero element. Apply the induction hypothesis to $\mathfrak m$ acting on $V$. We obtain the nonzero common-kernel subspace \begin{align*} W := \{v \in V : M(v)=0 \text{ for every } M \in \mathfrak m\}. \end{align*} This subspace is stable under the whole algebra $\mathfrak l$. Indeed, if $v\in W$, $A\in\mathfrak l$, and $M\in\mathfrak m$, then \begin{align*} M(A(v)) = A(M(v)) + [M,A]_{\operatorname{End}}(v)=0+0=0. \end{align*} The first term vanishes because $v\in W$, and the second term vanishes because $\mathfrak m$ is an ideal, so $[M,A]_{\operatorname{End}}\in\mathfrak m$. Choose $A_0\in\mathfrak l$ whose coset spans $\mathfrak l/\mathfrak m$, so $\mathfrak l=\mathfrak m\oplus F A_0$ as vector spaces. Since $W$ is invariant under $\mathfrak l$, the restriction \begin{align*} A_0|_W:W\to W \end{align*} is a nilpotent endomorphism. Hence it has a nonzero kernel. Pick $0\ne v\in W$ with $A_0(v)=0$. Then $\mathfrak m$ kills $v$ by definition of $W$, and $A_0$ kills $v$ by construction. Since every element of $\mathfrak l$ is $M+\lambda A_0$ for some $M\in\mathfrak m$ and $\lambda\in F$, every element of $\mathfrak l$ kills $v$.[/guided]

[step:Apply the invariant vector lemma to the adjoint representation]Let \begin{align*} \operatorname{ad}_{\mathfrak g}:\mathfrak g &\to \operatorname{End}_F(\mathfrak g) \\ x &\mapsto \operatorname{ad}_{\mathfrak g}(x) \end{align*} be the adjoint representation, and set \begin{align*} \mathfrak a := \operatorname{ad}_{\mathfrak g}(\mathfrak g) \subseteq \operatorname{End}_F(\mathfrak g). \end{align*} Since $\operatorname{ad}_{\mathfrak g}$ is a Lie algebra homomorphism, $\mathfrak a$ is a finite-dimensional Lie subalgebra of $\operatorname{End}_F(\mathfrak g)$. By hypothesis, every element of $\mathfrak a$ is nilpotent. If $\mathfrak g=0$, then $\mathfrak g$ is nilpotent. Assume henceforth that $\mathfrak g\ne 0$. Applying the invariant vector lemma with $V=\mathfrak g$ and $\mathfrak l=\mathfrak a$, there exists $0\ne z\in\mathfrak g$ such that \begin{align*} \operatorname{ad}_{\mathfrak g}(x)(z)=0 \end{align*} for every $x\in\mathfrak g$. Equivalently, \begin{align*} [x,z]_{\mathfrak g}=0 \end{align*} for every $x\in\mathfrak g$. Thus $z\in Z(\mathfrak g)$, where \begin{align*} Z(\mathfrak g):=\{u\in\mathfrak g:[x,u]_{\mathfrak g}=0\text{ for every }x\in\mathfrak g\}. \end{align*} Therefore $Z(\mathfrak g)\ne 0$.[/step]

[guided]The adjoint representation packages all brackets with a fixed element into a linear map. Define \begin{align*} \operatorname{ad}_{\mathfrak g}:\mathfrak g &\to \operatorname{End}_F(\mathfrak g) \\ x &\mapsto \operatorname{ad}_{\mathfrak g}(x), \end{align*} where $\operatorname{ad}_{\mathfrak g}(x)(y)=[x,y]_{\mathfrak g}$. Its image \begin{align*} \mathfrak a := \operatorname{ad}_{\mathfrak g}(\mathfrak g) \end{align*} is a finite-dimensional Lie subalgebra of $\operatorname{End}_F(\mathfrak g)$ because $\operatorname{ad}_{\mathfrak g}$ is a Lie algebra homomorphism. The hypothesis says exactly that every operator in this image is nilpotent. If $\mathfrak g=0$, there is nothing to prove. Otherwise, the invariant vector lemma applies to the nonzero vector space $V=\mathfrak g$ and the Lie algebra $\mathfrak l=\mathfrak a$. It gives an element $0\ne z\in\mathfrak g$ such that every operator in $\mathfrak a$ kills $z$. Written in adjoint notation, this is \begin{align*} \operatorname{ad}_{\mathfrak g}(x)(z)=0 \end{align*} for every $x\in\mathfrak g$. Unpacking the definition of $\operatorname{ad}_{\mathfrak g}(x)$ gives \begin{align*} [x,z]_{\mathfrak g}=0 \end{align*} for every $x\in\mathfrak g$. This is precisely the condition that $z$ belongs to the centre \begin{align*} Z(\mathfrak g):=\{u\in\mathfrak g:[x,u]_{\mathfrak g}=0\text{ for every }x\in\mathfrak g\}. \end{align*} Since $z\ne 0$, the centre is nonzero.[/guided]

[step:Show the quotient by the centre satisfies the same nilpotence hypothesis]Let \begin{align*} \pi:\mathfrak g &\to \mathfrak g/Z(\mathfrak g) \\ x &\mapsto x+Z(\mathfrak g) \end{align*} be the quotient map. For $\bar{x}:=\pi(x)\in \mathfrak g/Z(\mathfrak g)$, the adjoint endomorphism on the quotient is \begin{align*} \operatorname{ad}_{\mathfrak g/Z(\mathfrak g)}(\bar{x}):\mathfrak g/Z(\mathfrak g)&\to \mathfrak g/Z(\mathfrak g)\\ \pi(y)&\mapsto \pi([x,y]_{\mathfrak g}). \end{align*} This is well-defined because $Z(\mathfrak g)$ is an ideal of $\mathfrak g$. If $(\operatorname{ad}_{\mathfrak g}(x))^r=0$ for some $r\in\mathbb N$, then for every $y\in\mathfrak g$, \begin{align*} (\operatorname{ad}_{\mathfrak g/Z(\mathfrak g)}(\bar{x}))^r(\pi(y)) = \pi((\operatorname{ad}_{\mathfrak g}(x))^r(y)) = \pi(0) = 0. \end{align*} Hence every adjoint endomorphism of $\mathfrak g/Z(\mathfrak g)$ is nilpotent.[/step]

[guided]We now pass from $\mathfrak g$ to the quotient by its centre. Define the quotient map \begin{align*} \pi:\mathfrak g &\to \mathfrak g/Z(\mathfrak g) \\ x &\mapsto x+Z(\mathfrak g). \end{align*} The centre $Z(\mathfrak g)$ is an ideal because if $z\in Z(\mathfrak g)$ and $x\in\mathfrak g$, then $[x,z]_{\mathfrak g}=0\in Z(\mathfrak g)$. Therefore the quotient Lie algebra is defined. For $\bar{x}=\pi(x)$, the adjoint map in the quotient is \begin{align*} \operatorname{ad}_{\mathfrak g/Z(\mathfrak g)}(\bar{x}):\mathfrak g/Z(\mathfrak g)&\to \mathfrak g/Z(\mathfrak g)\\ \pi(y)&\mapsto \pi([x,y]_{\mathfrak g}). \end{align*} The formula is independent of representatives because we are quotienting by an ideal. The key point is that powers of the quotient adjoint map are induced by powers of the original adjoint map. If $(\operatorname{ad}_{\mathfrak g}(x))^r=0$, then for every $y\in\mathfrak g$, \begin{align*} (\operatorname{ad}_{\mathfrak g/Z(\mathfrak g)}(\bar{x}))^r(\pi(y)) = \pi((\operatorname{ad}_{\mathfrak g}(x))^r(y)) = \pi(0) = 0. \end{align*} Thus the nilpotence hypothesis survives passage to the quotient.[/guided]

[step:Induct on the dimension of the Lie algebra] We prove Engel's theorem by induction on $n:=\dim_F\mathfrak g$. If $n=0$, then $\mathfrak g=0$ is nilpotent. Suppose $n>0$, and assume the theorem holds for all Lie algebras over $F$ of dimension strictly smaller than $n$. By the previous steps, $Z(\mathfrak g)\ne 0$, and $\mathfrak g/Z(\mathfrak g)$ satisfies the same adjoint-nilpotence hypothesis. Since \begin{align*} \dim_F(\mathfrak g/Z(\mathfrak g)) < \dim_F\mathfrak g, \end{align*} the induction hypothesis implies that $\mathfrak g/Z(\mathfrak g)$ is nilpotent. [/step]

[step:Lift nilpotence from the central quotient to $\mathfrak g$] Define the lower central series of $\mathfrak g$ by \begin{align*} \gamma_1(\mathfrak g)&:=\mathfrak g,\\ \gamma_{k+1}(\mathfrak g)&:=[\mathfrak g,\gamma_k(\mathfrak g)]_{\mathfrak g} \end{align*} for $k\in\mathbb N$. Define the lower central series of $\mathfrak g/Z(\mathfrak g)$ analogously. For every $k\in\mathbb N$, \begin{align*} \gamma_k(\mathfrak g/Z(\mathfrak g))=\pi(\gamma_k(\mathfrak g)). \end{align*} This follows by induction on $k$. The case $k=1$ is the definition of $\pi$. If it holds for $k$, then \begin{align*} \gamma_{k+1}(\mathfrak g/Z(\mathfrak g)) &= [\mathfrak g/Z(\mathfrak g),\gamma_k(\mathfrak g/Z(\mathfrak g))]\\ &= [\pi(\mathfrak g),\pi(\gamma_k(\mathfrak g))]\\ &= \pi([\mathfrak g,\gamma_k(\mathfrak g)]_{\mathfrak g})\\ &= \pi(\gamma_{k+1}(\mathfrak g)). \end{align*} Since $\mathfrak g/Z(\mathfrak g)$ is nilpotent, there exists $c\in\mathbb N$ such that \begin{align*} \gamma_c(\mathfrak g/Z(\mathfrak g))=0. \end{align*} Therefore $\pi(\gamma_c(\mathfrak g))=0$, so \begin{align*} \gamma_c(\mathfrak g)\subseteq Z(\mathfrak g). \end{align*} It follows that \begin{align*} \gamma_{c+1}(\mathfrak g) = [\mathfrak g,\gamma_c(\mathfrak g)]_{\mathfrak g} = 0. \end{align*} Thus the lower central series of $\mathfrak g$ terminates at $0$, so $\mathfrak g$ is nilpotent. [/step]