[proofplan]
We first verify that the extended density $\widetilde r$ is normalized and that its $\Theta$-marginal is the desired posterior density, using the unbiasedness identity defining $Z$. We then write the actual proposal density on the enlarged space and compare the forward and reverse accepted move measures. The Metropolis-Hastings ratio on the enlarged space reduces exactly to the displayed particle marginal ratio because the auxiliary simulation densities cancel. Symmetry of the accepted move measure gives detailed balance, and adding the rejection mass on the diagonal gives invariance of the full kernel.
[/proofplan]
[step:Normalize the extended density and identify its marginal]
Let $X:=\Theta\times\mathcal U$, let $\mathcal X:=\mathcal T\otimes\mathcal A$, and let $\mu:=\rho\otimes\lambda$. Define the unnormalized extended weight $w:X\to[0,\infty)$ by
\begin{align*}
w(\theta,u)=p(\theta)\widehat Z(\theta,u)m(\theta,u).
\end{align*}
The functions $p$, $\widehat Z$, and $m$ are measurable and nonnegative, so $w$ is $\mathcal X$-measurable and nonnegative.
By Tonelli's theorem for nonnegative [measurable functions](/page/Measurable%20Functions) applied to $w$,
\begin{align*}
\int_X w(\theta,u)\,d\mu(\theta,u)=\int_\Theta p(\theta)\left(\int_{\mathcal U}\widehat Z(\theta,u)m(\theta,u)\,d\lambda(u)\right)d\rho(\theta).
\end{align*}
By the definition of $Z$, this becomes
\begin{align*}
\int_X w(\theta,u)\,d\mu(\theta,u)
=
\int_\Theta p(\theta)Z(\theta)\,d\rho(\theta)
=
C.
\end{align*}
Since $0<C<\infty$, the function $\widetilde r=C^{-1}w$ is a probability density with respect to $\mu$.
For $A\in\mathcal T$, Tonelli's theorem for nonnegative measurable functions applied to $\mathbb 1_A(\theta)\widetilde r(\theta,u)$ gives
\begin{align*}
\widetilde\pi(A\times\mathcal U)=C^{-1}\int_A p(\theta)\left(\int_{\mathcal U}\widehat Z(\theta,u)m(\theta,u)\,d\lambda(u)\right)d\rho(\theta).
\end{align*}
Using again the definition of $Z$,
\begin{align*}
\widetilde\pi(A\times\mathcal U)
=
C^{-1}\int_A p(\theta)Z(\theta)\,d\rho(\theta).
\end{align*}
Thus the $\Theta$-marginal of $\widetilde\pi$ is the stated measure $\pi(\,\cdot\mid y)$.
[guided]
The first task is to check that the object called an extended target is genuinely a probability measure. We work on the product measurable space
$X=\Theta\times\mathcal U$ with $\mathcal X=\mathcal T\otimes\mathcal A$ and reference measure $\mu=\rho\otimes\lambda$. Define
\begin{align*}
w:\Theta\times\mathcal U &\to [0,\infty)
\end{align*}
by
\begin{align*}
w(\theta,u)=p(\theta)\widehat Z(\theta,u)m(\theta,u).
\end{align*}
This function is measurable because it is a product of nonnegative measurable functions on the product space.
Since $w\ge 0$, Tonelli's theorem for nonnegative measurable functions applies without any prior integrability assumption. It gives
\begin{align*}
\int_{\Theta\times\mathcal U}w(\theta,u)\,d(\rho\otimes\lambda)(\theta,u)=\int_\Theta\left(\int_{\mathcal U}p(\theta)\widehat Z(\theta,u)m(\theta,u)\,d\lambda(u)\right)d\rho(\theta).
\end{align*}
The factor $p(\theta)$ does not depend on $u$, so it can be pulled outside the inner integral:
\begin{align*}
\int_{\Theta\times\mathcal U}w(\theta,u)\,d(\rho\otimes\lambda)(\theta,u)
=
\int_\Theta p(\theta)\left(\int_{\mathcal U}\widehat Z(\theta,u)m(\theta,u)\,d\lambda(u)\right)d\rho(\theta).
\end{align*}
The inner integral is exactly $Z(\theta)$ by hypothesis. Therefore
\begin{align*}
\int_{\Theta\times\mathcal U}w(\theta,u)\,d(\rho\otimes\lambda)(\theta,u)
=
\int_\Theta p(\theta)Z(\theta)\,d\rho(\theta)
=
C.
\end{align*}
Because $0<C<\infty$, dividing by $C$ produces a probability density:
\begin{align*}
\widetilde r(\theta,u)=C^{-1}w(\theta,u).
\end{align*}
The same computation also identifies the marginal. If $A\in\mathcal T$, then
\begin{align*}
\widetilde\pi(A\times\mathcal U)
=
\int_{A\times\mathcal U}\widetilde r(\theta,u)\,d(\rho\otimes\lambda)(\theta,u).
\end{align*}
Using Tonelli's theorem for nonnegative measurable functions again for the nonnegative integrand $\mathbb 1_A(\theta)\widetilde r(\theta,u)$,
\begin{align*}
\widetilde\pi(A\times\mathcal U)=C^{-1}\int_A p(\theta)\left(\int_{\mathcal U}\widehat Z(\theta,u)m(\theta,u)\,d\lambda(u)\right)d\rho(\theta).
\end{align*}
The inner integral is $Z(\theta)$, so
\begin{align*}
\widetilde\pi(A\times\mathcal U)
=
C^{-1}\int_A p(\theta)Z(\theta)\,d\rho(\theta).
\end{align*}
This proves that the extended target has exactly the desired $\theta$-marginal. The role of unbiasedness is precisely here: integrating out $u$ replaces the random estimate $\widehat Z(\theta,u)$ by the true normalizing factor $Z(\theta)$.
[/guided]
[/step]
[step:Write the enlarged proposal density and its reverse ratio]
Define the proposal density $\kappa:X\times X\to[0,\infty)$ by
\begin{align*}
\kappa((\theta,u),(\theta',u'))=q(\theta,\theta')m(\theta',u').
\end{align*}
Since $q$ is $\mathcal T\otimes\mathcal T$-measurable and $m$ is $\mathcal T\otimes\mathcal A$-measurable, the function $\kappa$ is $\mathcal X\otimes\mathcal X$-measurable. The acceptance function $\alpha:X\times X\to[0,1]$ is also $\mathcal X\otimes\mathcal X$-measurable, because it is obtained from measurable nonnegative functions by restricting to the measurable set where $p(\theta)\widehat Z(\theta,u)q(\theta,\theta')>0$, taking a quotient there, applying the continuous map $t\mapsto \min\{1,t\}$, and setting the value to $0$ on the complementary measurable set.
For each fixed $(\theta,u)\in X$,
\begin{align*}
\int_X \kappa((\theta,u),(\theta',u'))\,d\mu(\theta',u')
=
\int_\Theta q(\theta,\theta')\left(\int_{\mathcal U}m(\theta',u')\,d\lambda(u')\right)d\rho(\theta')
=
1.
\end{align*}
Thus $\kappa$ is a Markov proposal density on $X$.
When
\begin{align*}
p(\theta)\widehat Z(\theta,u)q(\theta,\theta')>0
\end{align*}
and $m(\theta,u)m(\theta',u')>0$, the Metropolis-Hastings reverse-over-forward ratio for the extended target density $\widetilde r$ and proposal density $\kappa$ is
\begin{align*}
\frac{\widetilde r(\theta',u')\kappa((\theta',u'),(\theta,u))}
{\widetilde r(\theta,u)\kappa((\theta,u),(\theta',u'))}
=
\frac{C^{-1}p(\theta')\widehat Z(\theta',u')m(\theta',u')q(\theta',\theta)m(\theta,u)}
{C^{-1}p(\theta)\widehat Z(\theta,u)m(\theta,u)q(\theta,\theta')m(\theta',u')}.
\end{align*}
The assumptions in this ratio computation imply that $p(\theta)>0$, $\widehat Z(\theta,u)>0$, $q(\theta,\theta')>0$, $m(\theta,u)>0$, and $m(\theta',u')>0$, so every denominator factor being cancelled is positive. Cancelling these common positive factors gives
\begin{align*}
\frac{\widetilde r(\theta',u')\kappa((\theta',u'),(\theta,u))}{\widetilde r(\theta,u)\kappa((\theta,u),(\theta',u'))}=\frac{p(\theta')\widehat Z(\theta',u')q(\theta',\theta)}{p(\theta)\widehat Z(\theta,u)q(\theta,\theta')}.
\end{align*}
Thus, on the part of $X\times X$ where the ordinary Metropolis-Hastings quotient is defined with positive denominator, the displayed acceptance probability is exactly the ordinary Metropolis-Hastings acceptance probability on the enlarged space. The remaining zero-factor cases are handled directly at the level of the accepted move density in the next step, where no division by $m(\theta,u)$ or $m(\theta',u')$ is used.
[/step]
[step:Prove symmetry of the accepted move measure]
Define the accepted move density $a:X\times X\to[0,\infty)$ by
\begin{align*}
a(x,y)=\widetilde r(x)\kappa(x,y)\alpha(x,y),
\end{align*}
where $x=(\theta,u)$ and $y=(\theta',u')$. We prove that
\begin{align*}
a(x,y)=a(y,x)
\end{align*}
for all $x,y\in X$.
Suppose first that
\begin{align*}
p(\theta)\widehat Z(\theta,u)q(\theta,\theta')>0.
\end{align*}
Then
\begin{align*}
a(x,y)
=
C^{-1}m(\theta,u)m(\theta',u')\min\{p(\theta)\widehat Z(\theta,u)q(\theta,\theta'),p(\theta')\widehat Z(\theta',u')q(\theta',\theta)\}.
\end{align*}
The right-hand side is symmetric in $x$ and $y$. If the reverse product $p(\theta')\widehat Z(\theta',u')q(\theta',\theta)$ is also positive, the same computation is valid in both directions, and hence $a(x,y)=a(y,x)$. If the reverse product is zero, the displayed formula gives $a(x,y)=0$; the zero-case argument below with $x$ and $y$ interchanged gives $a(y,x)=0$.
It remains to handle zero cases. If
\begin{align*}
p(\theta)\widehat Z(\theta,u)q(\theta,\theta')=0,
\end{align*}
then $\alpha(x,y)=0$ by definition, and therefore $a(x,y)=0$. If also
\begin{align*}
p(\theta')\widehat Z(\theta',u')q(\theta',\theta)=0,
\end{align*}
then the same argument gives $a(y,x)=0$.
Assume instead that
\begin{align*}
p(\theta')\widehat Z(\theta',u')q(\theta',\theta)>0.
\end{align*}
If $m(\theta,u)=0$, then $\kappa(y,x)=q(\theta',\theta)m(\theta,u)=0$, so $a(y,x)=0$. Suppose $m(\theta,u)>0$. Since
\begin{align*}
p(\theta)\widehat Z(\theta,u)q(\theta,\theta')=0,
\end{align*}
either $p(\theta)\widehat Z(\theta,u)=0$ or $q(\theta,\theta')=0$. If $p(\theta)\widehat Z(\theta,u)=0$, then the numerator in the reverse acceptance ratio is zero, so $\alpha(y,x)=0$ and hence $a(y,x)=0$. If $q(\theta,\theta')=0$ and $p(\theta)\widehat Z(\theta,u)>0$, then $p(\theta)\widehat Z(\theta,u)m(\theta,u)>0$. If also $m(\theta',u')>0$, then the positivity of $p(\theta')\widehat Z(\theta',u')q(\theta',\theta)$ gives $p(\theta')\widehat Z(\theta',u')m(\theta',u')>0$, so the symmetric support condition forces $q(\theta',\theta)=0$, contradicting the current assumption. Therefore $m(\theta',u')=0$, and then $\widetilde r(y)=0$, so $a(y,x)=0$. The symmetric argument covers the case with $x$ and $y$ interchanged. Hence $a(x,y)=a(y,x)$ for all $x,y\in X$.
[guided]
The key point is that detailed balance should be checked for the measure of accepted moves, not merely for the formal ratio. Write
\begin{align*}
x=(\theta,u),\qquad y=(\theta',u').
\end{align*}
The density of moves that start at $x$, propose $y$, and are accepted is
\begin{align*}
a(x,y)=\widetilde r(x)\kappa(x,y)\alpha(x,y).
\end{align*}
We want to prove that this density is symmetric in $x$ and $y$.
First consider the nondegenerate case
\begin{align*}
p(\theta)\widehat Z(\theta,u)q(\theta,\theta')>0.
\end{align*}
Then the acceptance probability is the minimum of $1$ and the reverse-over-forward ratio:
\begin{align*}
\alpha(x,y)
=
\min\left\{1,\frac{p(\theta')\widehat Z(\theta',u')q(\theta',\theta)}
{p(\theta)\widehat Z(\theta,u)q(\theta,\theta')}\right\}.
\end{align*}
Multiplying by the target density and the proposal density gives
\begin{align*}
a(x,y)
=
C^{-1}p(\theta)\widehat Z(\theta,u)m(\theta,u)q(\theta,\theta')m(\theta',u')
\min\left\{1,\frac{p(\theta')\widehat Z(\theta',u')q(\theta',\theta)}
{p(\theta)\widehat Z(\theta,u)q(\theta,\theta')}\right\}.
\end{align*}
Using the elementary identity $A\min\{1,B/A\}=\min\{A,B\}$ for $A>0$ and $B\ge 0$, with
\begin{align*}
A=p(\theta)\widehat Z(\theta,u)q(\theta,\theta')
\end{align*}
and
\begin{align*}
B=p(\theta')\widehat Z(\theta',u')q(\theta',\theta),
\end{align*}
we obtain
\begin{align*}
a(x,y)
=
C^{-1}m(\theta,u)m(\theta',u')\min\{p(\theta)\widehat Z(\theta,u)q(\theta,\theta'),p(\theta')\widehat Z(\theta',u')q(\theta',\theta)\}.
\end{align*}
This expression is unchanged when we swap $(\theta,u)$ with $(\theta',u')$. If the reverse product $p(\theta')\widehat Z(\theta',u')q(\theta',\theta)$ is positive, the same computation applies in the reverse direction, so this proves symmetry. If the reverse product is zero, the displayed formula gives $a(x,y)=0$; the zero-case argument below applied with $x$ and $y$ interchanged gives $a(y,x)=0$. Thus the forward-positive reverse-zero case is also covered.
Now we check that no division-by-zero case has been hidden. If
\begin{align*}
p(\theta)\widehat Z(\theta,u)q(\theta,\theta')=0,
\end{align*}
then the definition of the algorithm sets $\alpha(x,y)=0$, hence
\begin{align*}
a(x,y)=0.
\end{align*}
For symmetry, we must show that the reverse accepted density is also zero. If
\begin{align*}
p(\theta')\widehat Z(\theta',u')q(\theta',\theta)=0,
\end{align*}
then the reverse acceptance probability is also zero or the reverse proposal-target product is zero, so
\begin{align*}
a(y,x)=0.
\end{align*}
The only possible concern is the case
\begin{align*}
p(\theta')\widehat Z(\theta',u')q(\theta',\theta)>0.
\end{align*}
If $m(\theta,u)=0$, then the reverse proposal density contains the factor $m(\theta,u)$:
\begin{align*}
\kappa(y,x)=q(\theta',\theta)m(\theta,u)=0,
\end{align*}
so $a(y,x)=0$. If $m(\theta,u)>0$, then the equality
\begin{align*}
p(\theta)\widehat Z(\theta,u)q(\theta,\theta')=0
\end{align*}
forces either $p(\theta)\widehat Z(\theta,u)=0$ or $q(\theta,\theta')=0$. In the first case the reverse acceptance ratio has numerator zero, so $\alpha(y,x)=0$. In the second case, if $p(\theta)\widehat Z(\theta,u)m(\theta,u)>0$ and $p(\theta')\widehat Z(\theta',u')m(\theta',u')>0$, the symmetric support condition would force $q(\theta',\theta)=0$, contradicting the present assumption. If the second target factor is not positive because $m(\theta',u')=0$, then $\widetilde r(y)=0$, so $a(y,x)=0$. Thus every zero case also gives $a(y,x)=0$.
Therefore the accepted move density satisfies
\begin{align*}
a(x,y)=a(y,x)
\end{align*}
for all $x,y\in X$. This is the detailed balance identity for off-diagonal accepted moves.
[/guided]
[/step]
[step:Add the rejection probability and derive invariance]
Define the total acceptance probability $A:X\to[0,1]$ by
\begin{align*}
A(x)=\int_X \kappa(x,y)\alpha(x,y)\,d\mu(y).
\end{align*}
The Metropolis-Hastings kernel $K$ is therefore
\begin{align*}
K(x,B)=\int_B \kappa(x,y)\alpha(x,y)\,d\mu(y)+(1-A(x))\mathbb 1_B(x),
\qquad B\in\mathcal X.
\end{align*}
Let $f:X\to[0,\infty)$ be $\mathcal X$-measurable. Define $Kf:X\to[0,\infty]$ by
\begin{align*}
Kf(x)=\int_X f(y)\,K(x,dy).
\end{align*}
Using the definition of $K$,
\begin{align*}
\int_X Kf(x)\,d\widetilde\pi(x)
=
\int_X\int_X f(y)\kappa(x,y)\alpha(x,y)\,d\mu(y)\widetilde r(x)\,d\mu(x)
+
\int_X f(x)(1-A(x))\widetilde r(x)\,d\mu(x).
\end{align*}
By Tonelli's theorem for nonnegative measurable functions and the symmetry $a(x,y)=a(y,x)$,
\begin{align*}
\int_X\int_X f(y)a(x,y)\,d\mu(y)d\mu(x)=\int_X\int_X f(y)a(y,x)\,d\mu(y)d\mu(x).
\end{align*}
Renaming the variables in the right-hand side gives
\begin{align*}
\int_X\int_X f(y)a(y,x)\,d\mu(y)d\mu(x)
=
\int_X f(x)\left(\int_X a(x,y)\,d\mu(y)\right)d\mu(x).
\end{align*}
Since
\begin{align*}
\int_X a(x,y)\,d\mu(y)=\widetilde r(x)A(x),
\end{align*}
we obtain
\begin{align*}
\int_X Kf(x)\,d\widetilde\pi(x)
=
\int_X f(x)\widetilde r(x)A(x)\,d\mu(x)
+
\int_X f(x)(1-A(x))\widetilde r(x)\,d\mu(x).
\end{align*}
Combining the two terms,
\begin{align*}
\int_X Kf(x)\,d\widetilde\pi(x)
=
\int_X f(x)\widetilde r(x)\,d\mu(x)
=
\int_X f(x)\,d\widetilde\pi(x).
\end{align*}
Thus $\widetilde\pi K=\widetilde\pi$, so $\widetilde\pi$ is invariant for the Metropolis-Hastings kernel. Applying the marginal computation from the first step shows that the invariant $\Theta$-marginal is
\begin{align*}
\pi(d\theta\mid y)=C^{-1}p(\theta)Z(\theta)\,d\rho(\theta).
\end{align*}
This proves the theorem.
[/step]
