[guided]The new candidate should have length $N$, just like the old candidate, but it must begin at the new state $x^+$. The old first input $u_0^*$ has already been applied, so it cannot be reused as the first planned input for the next problem. The natural construction is therefore to shift the unused planned controls forward and append one terminal control at the end.
Define the new input tuple $(v_0,\dots,v_{N-1})$ by taking
$v_j := u_{j+1}^*$ for each $j \in \{0,\dots,N-2\}$, when this set is nonempty, and by setting
\begin{align*}
v_{N-1} := \kappa_f(x_N^*).
\end{align*}
This notation covers the edge case $N=1$: then there are no inherited controls $u_1^*,\dots,u_{N-1}^*$, and the new one-step input tuple is simply $(\kappa_f(x_1^*))$.
Now define the corresponding shifted state tuple $(y_0,\dots,y_N)$ by setting $y_j := x_{j+1}^*$ for each $j \in \{0,\dots,N-1\}$ and
\begin{align*}
y_N := F(x_N^*,\kappa_f(x_N^*)).
\end{align*}
We must check that these are admissible objects before checking the dynamics. For the inputs, if $j \leq N-2$, then $v_j=u_{j+1}^*\in U$ because the old input tuple belongs to $U^N$. The final input satisfies $v_{N-1}\in U$ because $x_N^*\in X_f$ and the terminal controller is a map $\kappa_f:X_f\to U$. Hence $(v_0,\dots,v_{N-1})\in U^N$.
For the states, if $j \leq N-1$, then $y_j=x_{j+1}^*\in X$ because the old state tuple belongs to $X^{N+1}$. The final state $y_N$ is also in $X$: indeed $x_N^*\in X_f\subset X$, $\kappa_f(x_N^*)\in U$, and the system map has codomain $X$, so
\begin{align*}
F(x_N^*,\kappa_f(x_N^*))\in X.
\end{align*}
Thus $(y_0,\dots,y_N)\in X^{N+1}$.[/guided]