[proofplan] We split the uniform error into a stochastic fluctuation term and a deterministic bias term. The bias is the difference between $Pg_{x,h}$ and $f(x)$, and it vanishes uniformly because $f$ is uniformly continuous while $K$ is compactly supported and has integral $1$. The stochastic term is controlled by a VC-type maximal inequality for the empirical process indexed by the bandwidth-scaled kernel class. The bandwidth assumptions force both the square-root and linear empirical-process rates to vanish, so the two pieces combine to give convergence in probability. [/proofplan]

[step:Decompose the estimator into empirical fluctuation and convolution bias] Let $P$ denote the common law of $X_1$, and for each $n \in \mathbb{N}$ let $P_n$ denote the empirical probability measure \begin{align*} P_n(B):=\frac{1}{n}\sum_{i=1}^n \mathbb{1}_B(X_i) \end{align*} for Borel sets $B \subset \mathbb{R}^d$. For $h>0$ and $x \in \mathbb{R}^d$, define the measurable function $g_{x,h}: \mathbb{R}^d \to \mathbb{R}$ by \begin{align*} g_{x,h}(u)=h^{-d}K\left(\frac{x-u}{h}\right). \end{align*} The kernel density estimator at bandwidth $h$ is therefore \begin{align*} \hat f_h(x) = P_n g_{x,h}. \end{align*} Also, \begin{align*} \mathbb{E}[\hat f_h(x)] = P g_{x,h}. \end{align*} Hence, for every compact set $A \subset \mathbb{R}^d$, \begin{align*} \|\hat f_h-f\|_{\infty,A} \leq \sup_{x \in A}|(P_n-P)g_{x,h}| + \sup_{x \in A}|Pg_{x,h}-f(x)|. \end{align*} It remains to prove that the two terms on the right converge to $0$ in probability, respectively deterministically, when $h=h_n$. [/step]

[step:Show that the convolution bias vanishes uniformly on compact sets]Let $R>0$ be such that $\operatorname{supp}K \subset \overline{B}(0,R)$. Since $K$ is bounded and compactly supported, $K \in L^1(\mathbb{R}^d,\mathcal{B}(\mathbb{R}^d),\mathcal{L}^d)$, and we define \begin{align*} M_K:=\int_{\mathbb{R}^d}|K(z)|\,d\mathcal{L}^d(z)<\infty. \end{align*} For $x \in \mathbb{R}^d$ and $h>0$, the substitution $z=(x-u)/h$, equivalently $u=x-hz$, transforms $d\mathcal{L}^d(u)$ into $h^d\,d\mathcal{L}^d(z)$ and gives \begin{align*} Pg_{x,h}=\int_{\mathbb{R}^d} h^{-d}K\left(\frac{x-u}{h}\right)f(u)\,d\mathcal{L}^d(u). \end{align*} After this change of variables, the same quantity is \begin{align*} Pg_{x,h}=\int_{\mathbb{R}^d}K(z)f(x-hz)\,d\mathcal{L}^d(z). \end{align*} Using $\int_{\mathbb{R}^d}K(z)\,d\mathcal{L}^d(z)=1$, we obtain \begin{align*} |Pg_{x,h}-f(x)|\leq \int_{\mathbb{R}^d}|K(z)|\,|f(x-hz)-f(x)|\,d\mathcal{L}^d(z). \end{align*} Since $K(z)=0$ for $z \notin \overline{B}(0,R)$, this bound becomes \begin{align*} |Pg_{x,h}-f(x)|\leq \int_{\overline{B}(0,R)}|K(z)|\,|f(x-hz)-f(x)|\,d\mathcal{L}^d(z). \end{align*} Because $f$ is uniformly continuous on $\mathbb{R}^d$, for every $\varepsilon>0$ there exists $\delta>0$ such that \begin{align*} |f(y)-f(x)|<\frac{\varepsilon}{1+M_K} \end{align*} whenever $|y-x|<\delta$. If $h>0$ satisfies \begin{align*} 0<h<\frac{\delta}{1+R} \end{align*} and $z \in \overline{B}(0,R)$, then $|x-hz-x|\leq hR<\delta$, and therefore \begin{align*} \sup_{x \in A}|Pg_{x,h}-f(x)| \leq \frac{\varepsilon}{1+M_K}M_K <\varepsilon. \end{align*} Thus \begin{align*} \sup_{x \in A}|Pg_{x,h}-f(x)| \to 0 \end{align*} as $h \downarrow 0$.[/step]

[guided]The deterministic term is the bias of the kernel estimator. We first rewrite it as an approximate identity acting on $f$. Let $R>0$ be chosen so that $\operatorname{supp}K \subset \overline{B}(0,R)$. Since $K$ is bounded and vanishes outside the compact set $\overline{B}(0,R)$, it is absolutely integrable with respect to [Lebesgue measure](/page/Lebesgue%20Measure), so the finite constant \begin{align*} M_K:=\int_{\mathbb{R}^d}|K(z)|\,d\mathcal{L}^d(z) \end{align*} is well-defined. For $x \in \mathbb{R}^d$ and $h>0$, compute the expectation using the density of $X_1$. The map $u \mapsto h^{-d}K((x-u)/h)$ is Borel measurable and bounded because $K$ is Borel measurable and bounded. Thus \begin{align*} Pg_{x,h} = \int_{\mathbb{R}^d} h^{-d}K\left(\frac{x-u}{h}\right)f(u)\,d\mathcal{L}^d(u). \end{align*} Now use the affine change of variables $z=(x-u)/h$, or $u=x-hz$. The Jacobian determinant has absolute value $h^d$, so \begin{align*} d\mathcal{L}^d(u)=h^d\,d\mathcal{L}^d(z). \end{align*} The whole domain $\mathbb{R}^d$ maps onto $\mathbb{R}^d$, and therefore \begin{align*} Pg_{x,h} = \int_{\mathbb{R}^d}K(z)f(x-hz)\,d\mathcal{L}^d(z). \end{align*} Since the kernel has unit integral, \begin{align*} f(x)=f(x)\int_{\mathbb{R}^d}K(z)\,d\mathcal{L}^d(z). \end{align*} Subtracting this identity from the previous display and applying the triangle inequality gives \begin{align*} |Pg_{x,h}-f(x)|\leq \int_{\mathbb{R}^d}|K(z)|\,|f(x-hz)-f(x)|\,d\mathcal{L}^d(z). \end{align*} Since $K(z)=0$ for $z \notin \overline{B}(0,R)$, the integral over $\mathbb{R}^d$ is the same as the integral over $\overline{B}(0,R)$, so \begin{align*} |Pg_{x,h}-f(x)|\leq \int_{\overline{B}(0,R)}|K(z)|\,|f(x-hz)-f(x)|\,d\mathcal{L}^d(z). \end{align*} The reason compact support matters is that $hz$ is uniformly small for every $z$ contributing to the integral. Let $\varepsilon>0$. [Uniform continuity](/page/Uniform%20Continuity) of $f$ on all of $\mathbb{R}^d$ gives a number $\delta>0$ such that \begin{align*} |y-x|<\delta \implies |f(y)-f(x)|<\frac{\varepsilon}{1+M_K} \end{align*} for all $x,y \in \mathbb{R}^d$. If $h>0$ satisfies \begin{align*} 0<h<\frac{\delta}{1+R} \end{align*} and $z \in \overline{B}(0,R)$, then \begin{align*} |x-hz-x|=h|z|\leq hR<\delta. \end{align*} Hence \begin{align*} \sup_{x \in A}|Pg_{x,h}-f(x)|\leq \int_{\overline{B}(0,R)}|K(z)|\frac{\varepsilon}{1+M_K}\,d\mathcal{L}^d(z). \end{align*} Using the definition of $M_K$, this gives \begin{align*} \sup_{x \in A}|Pg_{x,h}-f(x)|\leq \frac{\varepsilon M_K}{1+M_K}<\varepsilon. \end{align*} This proves that the bias converges to zero uniformly on $A$ as $h \downarrow 0$.[/guided]

[step:Control the empirical fluctuation by the VC kernel maximal inequality]Define, for each $h>0$ and compact $A \subset \mathbb{R}^d$, the function class \begin{align*} \mathcal{G}_{h,A}:=\{g_{x,h}: x \in A\}. \end{align*} We use the VC-subgraph and pointwise-measurability assumptions on $\mathcal K$. For fixed $h>0$ and compact $A$, the restricted scaled class $\mathcal{G}_{h,A}$ is pointwise measurable because pointwise measurability is inherited by restricting the index set and multiplying every function by the fixed positive scalar $h^{-d}$. The subgraph VC dimension is unchanged by multiplication by the positive scalar $h^{-d}$, because \begin{align*} \{(u,t):h^{-d}K((x-u)/h)>t\}=\{(u,t):K((x-u)/h)>h^d t\}, \end{align*} and the map $(u,t)\mapsto (u,h^dt)$ is a bijection of $\mathbb{R}^d\times\mathbb{R}$ preserving shattering relations. Let $B_K:=\sup_{z \in \mathbb{R}^d}|K(z)|<\infty$. Define the constant envelope $G_h: \mathbb{R}^d \to [0,\infty)$ by \begin{align*} G_h(u)=h^{-d}B_K. \end{align*} This is a measurable envelope for $\mathcal{G}_{h,A}$. Since $\mathcal{K}$ is VC-subgraph, there exist constants $a\geq e$ and $v\geq 1$, depending only on the VC characteristics of $\mathcal{K}$, such that for every probability measure $Q$ on $\mathbb{R}^d$ and every $0<\eta\leq 1$, \begin{align*} N\left(\eta\|G_h\|_{L^2(Q)},\mathcal{G}_{h,A},L^2(Q)\right) \leq \left(\frac{a}{\eta}\right)^v. \end{align*} This is the polynomial entropy hypothesis required by the outer-probability VC maximal inequality. The variance is bounded uniformly in $x \in A$. First, \begin{align*} P g_{x,h}^2=\int_{\mathbb{R}^d}h^{-2d}K\left(\frac{x-u}{h}\right)^2 f(u)\,d\mathcal{L}^d(u). \end{align*} Using the affine substitution $z=(x-u)/h$, equivalently $u=x-hz$, the measure transforms as $d\mathcal{L}^d(u)=h^d\,d\mathcal{L}^d(z)$ and the domain remains $\mathbb{R}^d$. Hence \begin{align*} P g_{x,h}^2=h^{-d}\int_{\mathbb{R}^d}K(z)^2f(x-hz)\,d\mathcal{L}^d(z). \end{align*} Since $f$ is bounded and $K$ is bounded with compact support, \begin{align*} P g_{x,h}^2\leq h^{-d}\|f\|_{\infty}\int_{\mathbb{R}^d}K(z)^2\,d\mathcal{L}^d(z). \end{align*} Set \begin{align*} C_{K,f}:= \|f\|_{\infty} \int_{\mathbb{R}^d}K(z)^2\,d\mathcal{L}^d(z)<\infty. \end{align*} Thus $\sup_{g\in\mathcal{G}_{h,A}}Pg^2\leq C_{K,f}h^{-d}$. We use the following VC empirical-process maximal inequality with Bernstein-Talagrand concentration, in the form of Theorem 2.14.1 in van der Vaart and Wellner's empirical-process theory together with the standard Bernstein-Talagrand tail integration: if a pointwise measurable class $\mathcal{F}$ has measurable envelope $F$, polynomial covering numbers \begin{align*} N\left(\eta\|F\|_{L^2(Q)},\mathcal{F},L^2(Q)\right)\leq \left(\frac{a}{\eta}\right)^v \end{align*} for all probability measures $Q$ and all $0<\eta\leq 1$, and if $\sup_{g\in\mathcal F}Pg^2\leq \sigma^2$, then the measurable [random variable](/page/Random%20Variable) $\sup_{g\in\mathcal F}|(P_n-P)g|$ satisfies \begin{align*} \sup_{g\in\mathcal F}|(P_n-P)g| = O_{\mathbb P}\left(\sqrt{\frac{\sigma^2\Lambda}{n}}\right) + O_{\mathbb P}\left(\frac{\|F\|_{\infty}\Lambda}{n}\right), \end{align*} where \begin{align*} \Lambda:=1+\log\left(\frac{a\|F\|_{\infty}}{\sigma}\right), \end{align*} and the implicit constants depend only on $a$ and $v$. Apply this result with $\mathcal F=\mathcal G_{h,A}$, $F=G_h$, and $\sigma^2=C_{K,f}h^{-d}$. The preceding paragraphs verify pointwise measurability, polynomial entropy, measurability of the envelope, and the variance bound. Moreover $\|G_h\|_{\infty}=B_Kh^{-d}$, so \begin{align*} \Lambda = 1+\log\left(\frac{aB_Kh^{-d}}{C_{K,f}^{1/2}h^{-d/2}}\right) \leq C_1\log(e/h) \end{align*} for a constant $C_1=C_1(a,B_K,C_{K,f},d)>0$ and all sufficiently small $h>0$. Therefore there is a constant $C_2=C_2(a,v,B_K,C_{K,f},d)>0$ such that along $h=h_n$, \begin{align*} \sup_{x \in A}|(P_n-P)g_{x,h}| = O_{\mathbb{P}}\left(\sqrt{\frac{\log(e/h)}{n h^d}}\right)+O_{\mathbb{P}}\left(\frac{\log(e/h)}{n h^d}\right) \end{align*} as $n \to \infty$ and $h=h_n \downarrow 0$.[/step]

[guided]The technically important point is that the empirical-process theorem is applied to the scaled class $\mathcal G_{h,A}$, so both the variance and the envelope depend on $h$. We verify the hypotheses from scratch. For fixed $h>0$ and compact $A \subset \mathbb{R}^d$, define \begin{align*} \mathcal{G}_{h,A}:=\{g_{x,h}:x\in A\}, \end{align*} where $g_{x,h}:\mathbb{R}^d\to\mathbb{R}$ is the measurable function \begin{align*} g_{x,h}(u)=h^{-d}K\left(\frac{x-u}{h}\right). \end{align*} We use an outer-probability version of the maximal inequality, so the restricted class $\mathcal G_{h,A}$ need not be separately shown to be pointwise measurable. The VC-subgraph property is still available for the scaled class: multiplication by $h^{-d}>0$ sends subgraphs to subgraphs after the vertical change of variables $t\mapsto h^dt$, and this change preserves the relevant shattering numbers. The pointwise-measurability property is also inherited by restricting the original index set and multiplying the resulting countable approximating subclass by $h^{-d}$. If \begin{align*} B_K:=\sup_{z\in\mathbb{R}^d}|K(z)|, \end{align*} then the constant function $G_h:\mathbb{R}^d\to[0,\infty)$ defined by \begin{align*} G_h(u)=h^{-d}B_K \end{align*} is a measurable envelope for $\mathcal G_{h,A}$, and \begin{align*} \|G_h\|_\infty=B_Kh^{-d}. \end{align*} Because the VC-subgraph property is preserved under multiplication by a positive scalar, the scaled class has polynomial entropy: there are constants $a\geq e$ and $v\geq 1$, depending only on the VC characteristics of the kernel class, such that for every probability measure $Q$ on $\mathbb{R}^d$ and every $0<\eta\leq 1$, \begin{align*} N\left(\eta\|G_h\|_{L^2(Q)},\mathcal{G}_{h,A},L^2(Q)\right) \leq \left(\frac{a}{\eta}\right)^v. \end{align*} It remains to compute the second-moment scale. For $x\in A$, \begin{align*} Pg_{x,h}^2=\int_{\mathbb{R}^d}h^{-2d}K\left(\frac{x-u}{h}\right)^2f(u)\,d\mathcal{L}^d(u). \end{align*} Use the affine substitution $z=(x-u)/h$, equivalently $u=x-hz$. This maps $\mathbb{R}^d$ onto $\mathbb{R}^d$ and transforms the measure by \begin{align*} d\mathcal{L}^d(u)=h^d\,d\mathcal{L}^d(z). \end{align*} Therefore \begin{align*} Pg_{x,h}^2=h^{-d}\int_{\mathbb{R}^d}K(z)^2f(x-hz)\,d\mathcal{L}^d(z). \end{align*} Since $f$ is bounded and $K$ is bounded with compact support, define \begin{align*} C_{K,f}:= \|f\|_{\infty} \int_{\mathbb{R}^d}K(z)^2\,d\mathcal{L}^d(z)<\infty. \end{align*} Then \begin{align*} \sup_{g\in\mathcal G_{h,A}}Pg^2\leq C_{K,f}h^{-d}. \end{align*} We now apply the VC empirical-process maximal inequality with Bernstein-Talagrand concentration, in the cited entropy form for pointwise measurable VC-type classes. Its hypotheses in this formulation are pointwise measurability, the measurable envelope, polynomial entropy, and the second-moment bound verified above. Applying it with $\mathcal F=\mathcal G_{h,A}$, $F=G_h$, and $\sigma^2=C_{K,f}h^{-d}$ gives the ordinary-probability estimate \begin{align*} \sup_{g\in\mathcal G_{h,A}}|(P_n-P)g| = O_{\mathbb P}\left(\sqrt{\frac{\sigma^2\Lambda}{n}}\right) + O_{\mathbb P}\left(\frac{\|G_h\|_\infty\Lambda}{n}\right), \end{align*} where \begin{align*} \Lambda:=1+\log\left(\frac{a\|G_h\|_\infty}{\sigma}\right). \end{align*} Substituting $\|G_h\|_\infty=B_Kh^{-d}$ and $\sigma=C_{K,f}^{1/2}h^{-d/2}$ yields \begin{align*} \Lambda = 1+\log\left(\frac{aB_Kh^{-d}}{C_{K,f}^{1/2}h^{-d/2}}\right) \leq C_1\log(e/h) \end{align*} for a constant $C_1=C_1(a,B_K,C_{K,f},d)>0$ and all sufficiently small $h>0$. This logarithmic factor is the entropy logarithm evaluated at the ratio of the envelope size to the standard-deviation scale. Substituting this bound and $\sigma^2=C_{K,f}h^{-d}$ into the maximal inequality gives \begin{align*} \sup_{x \in A}|(P_n-P)g_{x,h}| = O_{\mathbb{P}}\left(\sqrt{\frac{\log(e/h)}{n h^d}}\right)+O_{\mathbb{P}}\left(\frac{\log(e/h)}{n h^d}\right), \end{align*} with constants depending only on $a$, $v$, $B_K$, $C_{K,f}$, and $d$.[/guided]

[step:Use the bandwidth assumptions to force the stochastic term to vanish] Since $\log(1/h_n)=O(\log n)$, there is a constant $C_h>0$ and an integer $N_h \in \mathbb{N}$ such that \begin{align*} \log(e/h_n) \leq C_h \log n \end{align*} for all $n \geq N_h$. Therefore \begin{align*} \frac{\log(e/h_n)}{n h_n^d} \leq C_h\frac{\log n}{n h_n^d} \to 0 \end{align*} because $n h_n^d/\log n \to \infty$. It follows also that \begin{align*} \sqrt{\frac{\log(e/h_n)}{n h_n^d}}\to 0. \end{align*} The stochastic estimate from the previous step therefore gives \begin{align*} \sup_{x \in A}|(P_n-P)g_{x,h_n}| \xrightarrow{\mathbb{P}}0. \end{align*} This is ordinary convergence in probability because the pointwise-measurability hypothesis made the indexed supremum a measurable random variable in the maximal inequality. [/step]

[step:Combine the stochastic and bias estimates] For the fixed compact set $A \subset \mathbb{R}^d$, the decomposition from the first step gives \begin{align*} \|\hat f_{h_n}-f\|_{\infty,A} \leq \sup_{x \in A}|(P_n-P)g_{x,h_n}| + \sup_{x \in A}|Pg_{x,h_n}-f(x)|. \end{align*} The first term on the right converges to $0$ in probability by the VC empirical-process estimate and the bandwidth assumptions. The second term converges to $0$ deterministically because $h_n \downarrow 0$ and the convolution bias vanishes uniformly. Hence, by the preceding inequality, \begin{align*} \|\hat f_{h_n}-f\|_{\infty,A}\xrightarrow{\mathbb{P}}0. \end{align*} This proves the asserted uniform consistency on every compact set $A \subset \mathbb{R}^d$. [/step]