[guided]The expression we currently have for the $i$-th summand is
\begin{align*}
\Bigl\langle \int_a^s S(x)_{a,u}^{(i-1)} \otimes dx_u,\; \int_c^t S(y)_{c,v}^{(i-1)} \otimes dy_v \Bigr\rangle_{V^{\otimes i}}.
\end{align*}
We want to rewrite this as a double integral over $[a,s] \times [c,t]$ with the inner product passed *inside* the integrals. Why is this the right move? Because the eventual goal is to recognise the integrand as $k_{\phi_+}(x,y)_{u,v}$, which is itself an inner product at the lower level $V^{\otimes (i-1)}$ — precisely the level that survives once one tensor factor has been peeled off.
The tool is the multiplicative property of the Hilbert--Schmidt inner product on a tensor product: under the associativity isomorphism $V^{\otimes (j+1)} \cong V^{\otimes j} \otimes V$ given on simple tensors by $v_1 \otimes \cdots \otimes v_{j+1} \mapsto (v_1 \otimes \cdots \otimes v_j) \otimes v_{j+1}$ and extended bilinearly, the HS inner product satisfies
\begin{align*}
\langle A \otimes a,\, B \otimes b\rangle_{V^{\otimes (j+1)}} = \langle A, B\rangle_{V^{\otimes j}}\, \langle a, b\rangle_V \qquad \text{for } A, B \in V^{\otimes j},\ a, b \in V.
\end{align*}
This is the defining property of the HS inner product on a tensor product (and reduces to the standard Frobenius identity $\operatorname{tr}((A \otimes a)(B \otimes b)^*) = \operatorname{tr}(AB^*)\, a \cdot b$ in the matrix case).
We apply this with $j = i-1$, $A = S(x)_{a,u}^{(i-1)}$, $B = S(y)_{c,v}^{(i-1)}$, $a = dx_u$, $b = dy_v$:
\begin{align*}
\langle S(x)_{a,u}^{(i-1)} \otimes dx_u,\, S(y)_{c,v}^{(i-1)} \otimes dy_v\rangle_{V^{\otimes i}} = \langle S(x)_{a,u}^{(i-1)}, S(y)_{c,v}^{(i-1)}\rangle_{V^{\otimes (i-1)}}\, \langle dx_u, dy_v\rangle_V.
\end{align*}
This factorisation is *pointwise* in $(u,v)$.
To pass it through the integrals, recall that each Riemann--Stieltjes integral $\int_a^s S(x)_{a,u}^{(i-1)} \otimes dx_u$ is a *strong* limit of Riemann sums in the finite-dimensional space $V^{\otimes i}$. The HS inner product $\langle \cdot, \cdot\rangle_{V^{\otimes i}}$ is bilinear and continuous in each argument, so it commutes with the strong limit. Concretely, if $P_n$ is a sequence of Riemann sums tending to the integral on the $x$-side and $Q_n$ similarly on the $y$-side, then
\begin{align*}
\langle P_n, Q_n\rangle_{V^{\otimes i}} \to \Bigl\langle \int \cdots, \int \cdots \Bigr\rangle_{V^{\otimes i}},
\end{align*}
and the left-hand side, by bilinearity of the inner product on Riemann sums and the pointwise factorisation above, equals
\begin{align*}
\sum_{j,k} \langle S(x)_{a,u_j}^{(i-1)}, S(y)_{c,v_k}^{(i-1)}\rangle_{V^{\otimes (i-1)}}\, \langle x_{u_{j+1}} - x_{u_j}, y_{v_{k+1}} - y_{v_k}\rangle_V,
\end{align*}
which is itself a Riemann sum approximating
\begin{align*}
\int_a^s \int_c^t \langle S(x)_{a,u}^{(i-1)}, S(y)_{c,v}^{(i-1)}\rangle_{V^{\otimes (i-1)}}\, \langle dx_u, dy_v\rangle_V.
\end{align*}
Taking the limit identifies the inner product of the iterated integrals with this double Riemann--Stieltjes integral, justifying the passage.
Substituting back, we have
\begin{align*}
k_\phi(x,y)_{s,t} = \phi(0) + \sum_{i=1}^\infty \phi(i) \int_a^s \int_c^t \langle S(x)_{a,u}^{(i-1)}, S(y)_{c,v}^{(i-1)} \rangle_{V^{\otimes (i-1)}} \, \langle dx_u, dy_v \rangle_V.
\end{align*}
The sum and integrals are still in this order; swapping them is the next step's job, after we set up an integrable dominator.[/guided]