[proofplan]
The forward implication uses the smooth kernel to define $\widetilde R u$ by letting the distribution $u$ act on the [test function](/page/Test%20Function) $K(x,\cdot)$. Proper support is exactly what makes these test functions have a common compact support when $x$ ranges over a compact set, so the definition is valid and continuous for the strong topology on $\mathcal{D}'(U)$. For the converse, apply the assumed extension to point masses and set $K(x,y)=(\widetilde R\delta_y)(x)$; smooth dependence of $y \mapsto \delta_y$ and continuity of $\widetilde R$ imply that $K$ is smooth. Finally, testing against product functions identifies this smooth function with the Schwartz kernel of $R$, and uniqueness follows from the strong density of regularized test functions in $\mathcal{D}'(U)$.
[/proofplan]
[step:Use proper support to define the extension from a smooth kernel]
Assume first that $K_R$ is represented by a smooth function $K: U \times U \to \mathbb{C}$ and that $\operatorname{supp} K$ is proper over both factors. For each compact set $A \subset U$, define
\begin{align*}
B_A := \{y \in U : \text{there exists } x \in A \text{ with } (x,y) \in \operatorname{supp} K\}.
\end{align*}
Since the projection $\operatorname{supp} K \to U$ onto the first factor is proper, $B_A$ is compact in $U$.
For $u \in \mathcal{D}'(U)$, define $\widetilde R u: U \to \mathbb{C}$ by
\begin{align*}
(\widetilde R u)(x) := u(K(x,\cdot)).
\end{align*}
Here $K(x,\cdot): U \to \mathbb{C}$ denotes the smooth function $y \mapsto K(x,y)$. For each fixed $x \in U$, the set $\{y \in U : K(x,y) \ne 0\}$ has compact closure in $U$ by proper support, so $K(x,\cdot) \in C_c^\infty(U)$ and the distributional action $u(K(x,\cdot))$ is defined.
Let $\alpha \in \mathbb{N}_0^n$ be a multi-index. Since the map
\begin{align*}
x \mapsto D_x^\alpha K(x,\cdot)
\end{align*}
is a smooth map from $U$ into $C_c^\infty(U)$ locally with values in a fixed compact-support space, continuity and linearity of $u$ give
\begin{align*}
D_x^\alpha(\widetilde R u)(x) = u(D_x^\alpha K(x,\cdot)).
\end{align*}
Thus $\widetilde R u \in C^\infty(U)$.
[guided]
We start with the direction where the kernel is already a smooth function. The only possible obstruction to defining $(\widetilde R u)(x)$ by applying $u$ to $K(x,\cdot)$ is support: a distribution $u \in \mathcal{D}'(U)$ acts on compactly supported smooth test functions, not on arbitrary smooth functions.
Proper support removes this obstruction. Let $A \subset U$ be compact and define
\begin{align*}
B_A := \{y \in U : \text{there exists } x \in A \text{ with } (x,y) \in \operatorname{supp} K\}.
\end{align*}
This is the projection to the second factor of $\operatorname{supp} K \cap (A \times U)$. Since the projection $\operatorname{supp} K \to U$ onto the first factor is proper, the inverse image of the compact set $A$ is compact in $\operatorname{supp} K$, and therefore its projection $B_A$ is compact in $U$.
Now fix $x \in U$. Taking $A=\{x\}$ shows that the support of the function $K(x,\cdot): U \to \mathbb{C}$ is contained in the compact set $B_{\{x\}}$. Hence
\begin{align*}
K(x,\cdot) \in C_c^\infty(U).
\end{align*}
So for every distribution $u \in \mathcal{D}'(U)$ we may define
\begin{align*}
(\widetilde R u)(x) := u(K(x,\cdot)).
\end{align*}
We next prove that this is smooth in $x$. Let $\alpha \in \mathbb{N}_0^n$ be a multi-index. Since $K$ is smooth on $U \times U$, the partial derivative $D_x^\alpha K$ is again smooth, and for $x$ restricted to a compact set $A$, the functions $D_x^\alpha K(x,\cdot)$ all have support in the same compact set $B_A$. Therefore the map
\begin{align*}
x \mapsto D_x^\alpha K(x,\cdot)
\end{align*}
is a smooth map from $A$ into the [Fréchet space](/page/Fr%C3%A9chet%20Space) $C_{B_A}^\infty(U)$ of smooth functions supported in $B_A$. Since $u$ is a continuous linear functional on $C_c^\infty(U)$, composing this smooth test-function-valued map with $u$ gives a smooth scalar function. Differentiating through the continuous linear functional gives
\begin{align*}
D_x^\alpha(\widetilde R u)(x) = u(D_x^\alpha K(x,\cdot)).
\end{align*}
This proves $\widetilde R u \in C^\infty(U)$.
[/guided]
[/step]
[step:Prove continuity of the extension for the strong topology]
Let $A \subset U$ be compact and let $\alpha \in \mathbb{N}_0^n$. The corresponding $C^\infty(U)$ seminorm is
\begin{align*}
p_{A,\alpha}(f) := \sup_{x \in A} |D_x^\alpha f(x)|.
\end{align*}
For $u \in \mathcal{D}'(U)$, the previous step gives
\begin{align*}
p_{A,\alpha}(\widetilde R u) = \sup_{x \in A} |u(D_x^\alpha K(x,\cdot))|.
\end{align*}
Define
\begin{align*}
\mathcal{B}_{A,\alpha} := \{D_x^\alpha K(x,\cdot) : x \in A\} \subset C_c^\infty(U).
\end{align*}
The family $\mathcal{B}_{A,\alpha}$ is bounded in $C_c^\infty(U)$: all its elements are supported in the fixed compact set $B_A$, and for every multi-index $\beta \in \mathbb{N}_0^n$,
\begin{align*}
\sup_{x \in A}\sup_{y \in B_A}|D_y^\beta D_x^\alpha K(x,y)| < \infty
\end{align*}
because $A \times B_A$ is compact and $D_y^\beta D_x^\alpha K$ is continuous.
The strong topology on $\mathcal{D}'(U)$ is the topology of [uniform convergence](/page/Uniform%20Convergence) on bounded subsets of $C_c^\infty(U)$. Hence the map
\begin{align*}
u \mapsto \sup_{\eta \in \mathcal{B}_{A,\alpha}} |u(\eta)|
\end{align*}
is a continuous seminorm on $\mathcal{D}'(U)$, and
\begin{align*}
p_{A,\alpha}(\widetilde R u) \leq \sup_{\eta \in \mathcal{B}_{A,\alpha}} |u(\eta)|.
\end{align*}
Thus $\widetilde R: \mathcal{D}'(U) \to C^\infty(U)$ is continuous.
[/step]
[step:Verify that the extension agrees with the original operator on test functions]
Let $\phi \in C_c^\infty(U)$ and let $T_\phi \in \mathcal{D}'(U)$ be the associated [regular distribution](/page/Regular%20Distribution). For each $x \in U$,
\begin{align*}
(\widetilde R T_\phi)(x) = T_\phi(K(x,\cdot)).
\end{align*}
By the definition of $T_\phi$,
\begin{align*}
(\widetilde R T_\phi)(x) = \int_U K(x,y)\phi(y)\, d\mathcal{L}^n(y).
\end{align*}
Therefore, for every $\psi \in C_c^\infty(U)$,
\begin{align*}
T_{\widetilde R T_\phi}(\psi) = \int_U \psi(x)\left(\int_U K(x,y)\phi(y)\, d\mathcal{L}^n(y)\right)d\mathcal{L}^n(x).
\end{align*}
Since $\psi(x)K(x,y)\phi(y)$ has compact support in $U \times U$, [Fubini's theorem](/theorems/2961) applies and gives
\begin{align*}
T_{\widetilde R T_\phi}(\psi) = \int_{U \times U} K(x,y)\psi(x)\phi(y)\, d\mathcal{L}^{2n}(x,y).
\end{align*}
By the definition of the Schwartz kernel of $R$, this equals $K_R(\psi \otimes \phi)=(R\phi)(\psi)$. Hence the regular distribution associated to $\widetilde R(T_\phi)$ is $R\phi$.
[/step]
[step:Construct a smooth kernel from a continuous extension]
Conversely, assume that a continuous linear extension
\begin{align*}
\widetilde R: \mathcal{D}'(U) \to C^\infty(U)
\end{align*}
exists. For each $y \in U$, define the point-mass distribution $\delta_y \in \mathcal{D}'(U)$ by
\begin{align*}
\delta_y(\phi) := \phi(y)
\end{align*}
for all $\phi \in C_c^\infty(U)$. Define $K: U \times U \to \mathbb{C}$ by
\begin{align*}
K(x,y) := (\widetilde R\delta_y)(x).
\end{align*}
For every multi-index $\beta \in \mathbb{N}_0^n$, define the parameter derivative $\partial_y^\beta\delta_y \in \mathcal{D}'(U)$ by
\begin{align*}
(\partial_y^\beta\delta_y)(\phi) := D^\beta\phi(y)
\end{align*}
for all $\phi \in C_c^\infty(U)$. We verify the needed strong-topology smoothness. Let $A \subset U$ be compact, let $\gamma \in \mathbb{N}_0^n$, and let $\mathcal{B} \subset C_c^\infty(U)$ be bounded. By the structure of bounded sets in $C_c^\infty(U)$, there is a compact set $C \subset U$ such that every $\phi \in \mathcal{B}$ is supported in $C$, and for every multi-index $\rho \in \mathbb{N}_0^n$,
\begin{align*}
M_\rho := \sup_{\phi \in \mathcal{B}}\sup_{z \in C}|D^\rho\phi(z)| < \infty.
\end{align*}
Let $y \in A$ and let $h \in \mathbb{R}^n$ be a nonzero increment such that $y+h \in A$ and the line segment $[y,y+h] := \{y+th : 0 \leq t \leq 1\}$ is contained in $U$. For each coordinate vector $e_i \in \mathbb{R}^n$, [Taylor's theorem with integral remainder](/theorems/189) applied to the scalar function $t \mapsto \phi(y+t h_i e_i)$ gives
\begin{align*}
\left|\frac{\phi(y+h_i e_i)-\phi(y)}{h_i}-\partial_{x_i}\phi(y)\right| \leq |h_i|\sup_{z \in A_i}|\partial_{x_i}^2\phi(z)|
\end{align*}
whenever $h_i \ne 0$, where $A_i := \{y+t h_i e_i : y \in A, 0 \leq t \leq 1, |h_i| \leq r\}$ and $r>0$ is chosen so that $A_i \subset U$. Because $\mathcal{B}$ is bounded, the right-hand side is bounded above by $|h_i|M_{2e_i}$ uniformly in $\phi \in \mathcal{B}$. Thus the difference quotients of $y \mapsto \delta_y$ converge to $y \mapsto \partial_{y_i}\delta_y$ for every strong seminorm
\begin{align*}
q_{\mathcal{B}}(u) := \sup_{\phi \in \mathcal{B}} |u(\phi)|.
\end{align*}
Applying the same estimate to $D^\beta\phi$ in place of $\phi$ proves inductively that $y \mapsto \delta_y$ is $C^\infty$ as a map from $U$ into $\mathcal{D}'(U)$ with the strong topology, and its $\beta$-th parameter derivative is $y \mapsto \partial_y^\beta\delta_y$.
Since $\widetilde R$ is continuous linear, the map
\begin{align*}
y \mapsto \widetilde R(\delta_y)
\end{align*}
is smooth from $U$ into $C^\infty(U)$. We now justify the passage from $C^\infty(U)$-valued smoothness to joint smoothness on $U \times U$. Let $F_\beta: U \to C^\infty(U)$ denote the smooth map
\begin{align*}
F_\beta(y) := \widetilde R(\partial_y^\beta\delta_y).
\end{align*}
For each compact set $A \subset U$ and multi-index $\alpha \in \mathbb{N}_0^n$, the seminorm
\begin{align*}
p_{A,\alpha}(f) := \sup_{x \in A}|D_x^\alpha f(x)|
\end{align*}
is continuous on $C^\infty(U)$. Continuity of $F_\beta$ for this seminorm means that, whenever $y_j \to y$ in $U$,
\begin{align*}
\sup_{x \in A}|D_x^\alpha F_\beta(y_j)(x)-D_x^\alpha F_\beta(y)(x)| \to 0.
\end{align*}
Since each $F_\beta(y)$ is a smooth function of $x$, this proves that $(x,y) \mapsto D_x^\alpha F_\beta(y)(x)$ is continuous on $U \times U$: local uniform convergence in the $x$ variable gives continuity in $y$, and smoothness of each slice gives continuity in $x$. Differentiating the smooth $C^\infty(U)$-valued map $F_0(y)=\widetilde R(\delta_y)$ in the $y$ variable gives $F_\beta(y)$, so for all multi-indices $\alpha,\beta \in \mathbb{N}_0^n$,
\begin{align*}
D_x^\alpha D_y^\beta K(x,y) = D_x^\alpha\bigl(\widetilde R(\partial_y^\beta\delta_y)\bigr)(x).
\end{align*}
The preceding continuity argument applies to the right-hand side. Hence every mixed partial derivative exists and is continuous on $U \times U$, so $K \in C^\infty(U \times U)$.
[guided]
The converse begins by asking what the kernel should be. If $R$ really were given by a smooth kernel $K$, then applying $R$ to a point mass at $y$ would extract the $y$-slice of the kernel:
\begin{align*}
(\widetilde R\delta_y)(x) = K(x,y).
\end{align*}
This motivates the definition
\begin{align*}
K(x,y) := (\widetilde R\delta_y)(x).
\end{align*}
We must prove that this function is smooth in both variables. First define the point-mass distribution precisely: for $y \in U$, the distribution $\delta_y \in \mathcal{D}'(U)$ is
\begin{align*}
\delta_y(\phi) := \phi(y)
\end{align*}
for all $\phi \in C_c^\infty(U)$. For a multi-index $\beta \in \mathbb{N}_0^n$, define the parameter derivative $\partial_y^\beta\delta_y$ by
\begin{align*}
(\partial_y^\beta\delta_y)(\phi) := D^\beta\phi(y).
\end{align*}
This notation refers to differentiating the parameter $y$, not to the [distributional derivative](/page/Distributional%20Derivative) in the test-function variable.
We now verify the smoothness of $y \mapsto \delta_y$ in the strong topology. The strong topology on $\mathcal{D}'(U)$ is generated by seminorms
\begin{align*}
q_{\mathcal{B}}(u) := \sup_{\phi \in \mathcal{B}} |u(\phi)|,
\end{align*}
where $\mathcal{B} \subset C_c^\infty(U)$ is bounded. A bounded set $\mathcal{B}$ in $C_c^\infty(U)$ is contained in one compact-support space $C_C^\infty(U)$ for some compact set $C \subset U$, and all derivatives of all functions in $\mathcal{B}$ are uniformly bounded on $C$. Fix a compact set $A \subset U$ and choose $r>0$ so that the closed $r$-neighbourhood of $A$ is contained in $U$. For $y \in A$, $|h_i|<r$, and a coordinate vector $e_i$, [Taylor's theorem](/theorems/827) with integral remainder gives
\begin{align*}
\left|\frac{\phi(y+h_i e_i)-\phi(y)}{h_i}-\partial_{x_i}\phi(y)\right| \leq |h_i|\sup_{z \in A_i}|\partial_{x_i}^2\phi(z)|,
\end{align*}
where $A_i := \{y+t h_i e_i : y \in A, 0 \leq t \leq 1, |h_i|<r\}$. The right-hand side is bounded by $|h_i|M_{2e_i}$ uniformly for $\phi \in \mathcal{B}$. Hence the difference quotients converge with respect to every seminorm $q_{\mathcal{B}}$. Repeating the same estimate after replacing $\phi$ by $D^\beta\phi$ proves that
\begin{align*}
y \mapsto \delta_y
\end{align*}
is a $C^\infty$ map from $U$ into $\mathcal{D}'(U)$ with the strong topology, with $\beta$-th parameter derivative $y \mapsto \partial_y^\beta\delta_y$.
Now use the continuity and linearity of $\widetilde R$. A continuous [linear map](/page/Linear%20Map) between locally convex spaces sends smooth parameter maps to smooth parameter maps. Hence
\begin{align*}
y \mapsto \widetilde R(\delta_y)
\end{align*}
is a smooth map from $U$ into $C^\infty(U)$. We still have to explain why this gives joint smoothness in $(x,y)$, rather than merely smoothness in $y$ after evaluating at a fixed $x$. For a multi-index $\beta$, define
\begin{align*}
F_\beta: U \to C^\infty(U), \qquad F_\beta(y) := \widetilde R(\partial_y^\beta\delta_y).
\end{align*}
If $A \subset U$ is compact and $\alpha \in \mathbb{N}_0^n$, continuity of $F_\beta$ for the seminorm $p_{A,\alpha}(f)=\sup_{x \in A}|D_x^\alpha f(x)|$ says that $D_x^\alpha F_\beta(y_j)$ converges uniformly on $A$ whenever $y_j \to y$. This local uniform convergence in $x$, together with the ordinary smoothness of each slice $F_\beta(y):U\to\mathbb{C}$, proves continuity of
\begin{align*}
(x,y) \mapsto D_x^\alpha F_\beta(y)(x).
\end{align*}
Therefore differentiating in $y$ and then evaluating in $x$ gives
\begin{align*}
D_x^\alpha D_y^\beta K(x,y) = D_x^\alpha\bigl(\widetilde R(\partial_y^\beta\delta_y)\bigr)(x),
\end{align*}
and the right-hand side is continuous in $(x,y)$. Thus every mixed derivative exists and is continuous, so $K \in C^\infty(U \times U)$.
[/guided]
[/step]
[step:Identify the constructed function with the Schwartz kernel of $R$]
Let $\phi,\psi \in C_c^\infty(U)$. We first justify the distribution-valued integral used below. The map
\begin{align*}
y \mapsto \phi(y)\delta_y
\end{align*}
from $U$ into $\mathcal{D}'(U)$ has compact support contained in $\operatorname{supp}\phi$ and is continuous for the strong topology by the preceding step. Because its support is compact and $\mathcal{D}'(U)$ is a complete locally convex space for the strong topology, the usual weak, equivalently Pettis, integral over $U$ exists. Define $I_\phi \in \mathcal{D}'(U)$ by the requirement that every continuous linear functional $\Lambda:(\mathcal{D}'(U))_\beta\to\mathbb{C}$ satisfy
\begin{align*}
\Lambda(I_\phi)=\int_U \phi(y)\Lambda(\delta_y)\, d\mathcal{L}^n(y).
\end{align*}
Testing this identity on the continuous linear functionals $u\mapsto u(\eta)$, with $\eta\in C_c^\infty(U)$, gives
\begin{align*}
I_\phi(\eta) = \int_U \phi(y)\delta_y(\eta)\, d\mathcal{L}^n(y)=\int_U \phi(y)\eta(y)\, d\mathcal{L}^n(y)=T_\phi(\eta).
\end{align*}
Hence $I_\phi=T_\phi$ in $\mathcal{D}'(U)$.
Let $x \in U$. The scalar map $E_x: C^\infty(U) \to \mathbb{C}$, $E_x(f)=f(x)$, is continuous, so $E_x\circ \widetilde R: \mathcal{D}'(U) \to \mathbb{C}$ is continuous linear. Applying the defining property of the Pettis integral with $\Lambda=E_x\circ\widetilde R$ gives
\begin{align*}
(\widetilde R T_\phi)(x) = \int_U \phi(y)(\widetilde R\delta_y)(x)\, d\mathcal{L}^n(y).
\end{align*}
By the definition of $K$, this is
\begin{align*}
(\widetilde R T_\phi)(x) = \int_U \phi(y)K(x,y)\, d\mathcal{L}^n(y).
\end{align*}
Therefore
\begin{align*}
(R\phi)(\psi) = T_{\widetilde R T_\phi}(\psi).
\end{align*}
Using the previous formula and Fubini's theorem, whose hypotheses hold because $\psi(x)\phi(y)K(x,y)$ is smooth and compactly supported on $U \times U$, we obtain
\begin{align*}
(R\phi)(\psi) = \int_{U \times U} K(x,y)\psi(x)\phi(y)\, d\mathcal{L}^{2n}(x,y).
\end{align*}
Thus the Schwartz kernel of $R$ is represented by the smooth function $K$, because $K_R$ and the regular distribution induced by $K$ agree on all product test functions $\psi \otimes \phi$, and finite sums of such product test functions are dense in $C_c^\infty(U \times U)$.
[/step]
[step:Prove uniqueness of the continuous extension]
Let $\widetilde R_1,\widetilde R_2: \mathcal{D}'(U) \to C^\infty(U)$ be continuous linear extensions of $R$. Define
\begin{align*}
S := \widetilde R_1 - \widetilde R_2.
\end{align*}
Then $S: \mathcal{D}'(U) \to C^\infty(U)$ is continuous linear, and $S(T_\phi)=0$ for every $\phi \in C_c^\infty(U)$.
We verify the density statement needed here. Let $(K_j)_{j\in\mathbb{N}}$ be a compact exhaustion of $U$ such that $K_j \subset K_{j+1}^{\circ}$ and $\bigcup_{j=1}^{\infty}K_j=U$. Choose cutoff functions $\theta_j \in C_c^\infty(U)$ with $\theta_j=1$ on a neighbourhood of $K_j$. Choose a [standard mollifier](/page/Standard%20Mollifier) $\eta \in C_c^\infty(B(0,1))$ with
\begin{align*}
\int_{\mathbb{R}^n}\eta(z)\,d\mathcal{L}^n(z)=1,
\end{align*}
and set $\eta_\varepsilon(z)=\varepsilon^{-n}\eta(z/\varepsilon)$. For $u \in \mathcal{D}'(U)$, the product $\theta_j u$ has compact support in $U$; extend it by zero to a compactly supported distribution on $\mathbb{R}^n$. If $0<\varepsilon<\operatorname{dist}(\operatorname{supp}\theta_j,\mathbb{R}^n\setminus U)$, then
\begin{align*}
\phi_{j,\varepsilon}:=\eta_\varepsilon*(\theta_j u)
\end{align*}
is a function in $C_c^\infty(U)$.
We order the pairs $(j,\varepsilon)$ by declaring $(j,\varepsilon)$ large when $j$ is large and $\varepsilon$ is small subject to the above support condition. Let $\mathcal{B}\subset C_c^\infty(U)$ be bounded. Then there is a compact set $C\subset U$ such that every $\rho\in\mathcal{B}$ is supported in $C$, and all derivatives of all $\rho\in\mathcal{B}$ are uniformly bounded on $C$. Choose $j$ so large that $C\subset K_j$ and $\theta_j=1$ on a neighbourhood of $C$. For such $j$, and for all sufficiently small $\varepsilon$, duality for convolution gives
\begin{align*}
T_{\phi_{j,\varepsilon}}(\rho)=u(\theta_j(\check\eta_\varepsilon*\rho))
\end{align*}
for every $\rho\in\mathcal{B}$, where $\check\eta_\varepsilon(z):=\eta_\varepsilon(-z)$. Since $\theta_j=1$ near $C$ and $\check\eta_\varepsilon*\rho\to\rho$ in $C_c^\infty(U)$ uniformly for $\rho\in\mathcal{B}$, continuity of $u$ on the fixed compact-support test-function space gives
\begin{align*}
\sup_{\rho\in\mathcal{B}}|T_{\phi_{j,\varepsilon}}(\rho)-u(\rho)|\to 0.
\end{align*}
This is exactly convergence uniformly on the bounded set $\mathcal{B}$, so $T_{\phi_{j,\varepsilon}} \to u$ strongly in $\mathcal{D}'(U)$.
By continuity of $S$,
\begin{align*}
S(u) = \lim_{(j,\varepsilon)} S(T_{\phi_{j,\varepsilon}}) = 0
\end{align*}
in $C^\infty(U)$. Thus $\widetilde R_1(u)=\widetilde R_2(u)$ for every $u \in \mathcal{D}'(U)$, proving uniqueness.
Combining the forward construction, the converse construction, and uniqueness proves the characterization.
[/step]