Let $n,d\to\infty$ with $n/d\to\delta\in(0,\infty)$. Let $A\in\mathbb R^{n\times d}$ have independent entries $A_{ij}\sim \mathcal N(0,1/n)$. Let deterministic or random vectors $\theta\in\mathbb R^d$ and $w\in\mathbb R^n$ be independent of $A$ and satisfy the following empirical convergence assumptions: for every pseudo-Lipschitz function $\varphi:\mathbb R\to\mathbb R$ of order $2$,
\begin{align*}
\frac{1}{d}\sum_{i=1}^d \varphi(\theta_i) \xrightarrow{\mathbb P} \mathbb E[\varphi(\Theta)]
\end{align*}
for some $\Theta\in L^2$, and
\begin{align*}
\frac{1}{n}\sum_{i=1}^n \varphi(w_i) \xrightarrow{\mathbb P} \mathbb E[\varphi(W)]
\end{align*}
for $W\sim\mathcal N(0,\sigma^2)$. Assume $W$ is independent of $\Theta$. These assumptions include convergence of the empirical second moments to $\mathbb E[\Theta^2]$ and $\sigma^2$.
Consider the linear model $y=A\theta+w$ and the AMP recursion initialized by $x_0=0$, $r_{-1}=0$, and $r_0=y$. For separable denoisers $\eta_t:\mathbb R\to\mathbb R$ that are Lipschitz and differentiable except on a Lebesgue-null set, define
\begin{align*}
x_{t+1,i} = \eta_t((A^\top r_t+x_t)_i).
\end{align*}
The residual and Onsager coefficient are
\begin{align*}
r_t = y-Ax_t+b_t r_{t-1}
\end{align*}
and
\begin{align*}
b_t = \frac{1}{n}\sum_{i=1}^d \eta_{t-1}'((A^\top r_{t-1}+x_{t-1})_i),
\end{align*}
with $b_0=0$. Since a Lipschitz denoiser is differentiable almost everywhere, changing $\eta_{t-1}'$ on its exceptional set does not affect the limiting Onsager term. Let $\tau_0^2=\sigma^2+\delta^{-1}\mathbb E[\Theta^2]$, and define $(\tau_t^2)_{t\ge0}$ by the state-evolution recursion
\begin{align*}
\tau_{t+1}^2=\sigma^2+\frac{1}{\delta}\mathbb E\left[(\eta_t(\Theta+\tau_t Z)-\Theta)^2\right],
\end{align*}
where $Z\sim\mathcal N(0,1)$ is independent of $\Theta$.
For each fixed iteration $t$ and every pseudo-Lipschitz [test function](/page/Test%20Function) $\psi:\mathbb R^3\to\mathbb R$ of order $2$,
\begin{align*}
\frac{1}{d}\sum_{i=1}^d \psi\left(\theta_i,(A^\top r_t+x_t)_i,x_{t+1,i}\right)
\xrightarrow{\mathbb P}
\mathbb E\left[\psi\left(\Theta,\Theta+\tau_t Z,\eta_t(\Theta+\tau_t Z)\right)\right].
\end{align*}
In particular, the empirical law of the signal coordinate, AMP effective observation, and AMP estimate converges weakly in probability to the law of
\begin{align*}
(\Theta,\Theta+\tau_t Z,\eta_t(\Theta+\tau_t Z)),
\end{align*}
for each fixed $t$.