Let $n\in\mathbb{N}$, let $T>0$, let $U$ be a nonempty control set, and let $f:\mathbb{R}^n\times U\to\mathbb{R}^n$ and $\ell:\mathbb{R}^n\times U\to\mathbb{R}$ be continuous maps. Let $g:\mathbb{R}^n\to\mathbb{R}$ be the terminal cost, and suppose the value function $V\in C^1([0,T]\times\mathbb{R}^n)$ satisfies the terminal condition $V(T,x)=g(x)$ for every $x\in\mathbb{R}^n$. Suppose that, for every $(t,x)\in [0,T)\times\mathbb{R}^n$ and every sufficiently small $h>0$ with $t+h\leq T$, there is an admissible-control class $\mathcal{A}_{t,t+h}$ on $[t,t+h]$ such that each $u\in\mathcal{A}_{t,t+h}$ has a controlled trajectory $y_u\in C^1([t,t+h];\mathbb{R}^n)$ satisfying $y_u(t)=x$ and

\begin{align*} \dot y_u(s)=f(y_u(s),u(s)) \end{align*}

for $s\in[t,t+h]$, and the dynamic programming principle holds:

\begin{align*} V(t,x)=\inf_{u\in\mathcal{A}_{t,t+h}}\left\{\int_t^{t+h}\ell(y_u(s),u(s))\,d\mathcal{L}^1(s)+V(t+h,y_u(t+h))\right\}. \end{align*}

Assume that for each $a\in U$, the constant control $u_a:[t,t+h]\to U$ defined by $u_a(s)=a$ is admissible for all sufficiently small $h>0$. Assume also the following first-order approximate-minimizer consistency: for each $(t,x)\in[0,T)\times\mathbb{R}^n$ there is a remainder $r_{t,x}:(0,T-t]\to\mathbb{R}$ with $r_{t,x}(h)=o(h)$ as $h\downarrow 0$ such that, for all sufficiently small $h>0$,

\begin{align*} 0\geq h\left\{\partial_t V(t,x)+\inf_{a\in U}\bigl(\ell(x,a)+\nabla_x V(t,x)\cdot f(x,a)\bigr)\right\}+r_{t,x}(h). \end{align*}

Define the Hamiltonian $H:\mathbb{R}^n\times\mathbb{R}^n\to\mathbb{R}$ by

\begin{align*} H(x,p)=\inf_{a\in U}\{\ell(x,a)+p\cdot f(x,a)\}. \end{align*}

Then, for every $(t,x)\in [0,T)\times\mathbb{R}^n$,

\begin{align*} \partial_t V(t,x)+H(x,\nabla_x V(t,x))=0. \end{align*}

Equivalently,

\begin{align*} -\partial_t V(t,x)=\inf_{a\in U}\{\ell(x,a)+\nabla_x V(t,x)\cdot f(x,a)\}. \end{align*}

Moreover,

\begin{align*} V(T,x)=g(x) \end{align*}

for every $x\in\mathbb{R}^n$.