[proofplan]
We first rewrite the self-consistent equation as a quadratic equation for the Stieltjes transform. The large-$z$ condition selects the correct algebraic branch of the square root. We then compute the upper half-plane boundary values of that branch on the real axis and apply the Stieltjes inversion formula to recover the absolutely continuous density.
[/proofplan]
[step:Convert the self-consistent equation into a quadratic equation for $m_\gamma$]
Fix $z \in \mathbb{C}\setminus[0,\infty)$. Since the displayed equation contains the reciprocal terms
\begin{align*}
\frac{1}{m_\gamma(z)}
\end{align*}
and
\begin{align*}
\frac{1}{1+\gamma m_\gamma(z)},
\end{align*}
the hypotheses imply
\begin{align*}
m_\gamma(z) \neq 0.
\end{align*}
Also,
\begin{align*}
1+\gamma m_\gamma(z) \neq 0.
\end{align*}
Multiplying
\begin{align*}
z = -\frac{1}{m_\gamma(z)} + \frac{1}{1+\gamma m_\gamma(z)}
\end{align*}
by $m_\gamma(z)(1+\gamma m_\gamma(z))$ gives
\begin{align*}
z m_\gamma(z)(1+\gamma m_\gamma(z)) = -(1+\gamma m_\gamma(z))+m_\gamma(z).
\end{align*}
Therefore
\begin{align*}
z m_\gamma(z)(1+\gamma m_\gamma(z)) = -1+(1-\gamma)m_\gamma(z).
\end{align*}
Expanding and moving every term to the left, we obtain
\begin{align*}
\gamma z m_\gamma(z)^2 +(z+\gamma-1)m_\gamma(z)+1=0.
\end{align*}
Thus $m_\gamma(z)$ satisfies the quadratic equation
\begin{align*}
\gamma z w^2+(z+\gamma-1)w+1=0
\end{align*}
with unknown $w \in \mathbb{C}$.
[guided]
The purpose of this step is to turn the implicit equation into an algebraic formula. Fix $z \in \mathbb{C}\setminus[0,\infty)$. Because the equation contains the reciprocal term
\begin{align*}
\frac{1}{m_\gamma(z)},
\end{align*}
it already asserts that
\begin{align*}
m_\gamma(z) \neq 0.
\end{align*}
It also asserts that
\begin{align*}
1+\gamma m_\gamma(z) \neq 0.
\end{align*}
Therefore multiplication by $m_\gamma(z)(1+\gamma m_\gamma(z))$ is legitimate.
Starting from
\begin{align*}
z = -\frac{1}{m_\gamma(z)} + \frac{1}{1+\gamma m_\gamma(z)},
\end{align*}
we multiply both sides by $m_\gamma(z)(1+\gamma m_\gamma(z))$. This gives
\begin{align*}
z m_\gamma(z)(1+\gamma m_\gamma(z)) = -\frac{m_\gamma(z)(1+\gamma m_\gamma(z))}{m_\gamma(z)}+\frac{m_\gamma(z)(1+\gamma m_\gamma(z))}{1+\gamma m_\gamma(z)}.
\end{align*}
Canceling the two non-zero denominator factors gives
\begin{align*}
z m_\gamma(z)(1+\gamma m_\gamma(z)) = -(1+\gamma m_\gamma(z))+m_\gamma(z).
\end{align*}
Thus
\begin{align*}
z m_\gamma(z)(1+\gamma m_\gamma(z)) = -1+(1-\gamma)m_\gamma(z).
\end{align*}
Expanding the product on the left-hand side yields
\begin{align*}
z m_\gamma(z)+\gamma z m_\gamma(z)^2 = -1+(1-\gamma)m_\gamma(z).
\end{align*}
Moving every term to the left gives
\begin{align*}
\gamma z m_\gamma(z)^2 +(z+\gamma-1)m_\gamma(z)+1=0.
\end{align*}
Thus, for each admissible $z$, the value $m_\gamma(z)$ is one of the two roots of the quadratic polynomial
\begin{align*}
w \mapsto \gamma z w^2+(z+\gamma-1)w+1.
\end{align*}
[/guided]
[/step]
[step:Select the branch compatible with $z m_\gamma(z)\to -1$]
For $z \in \mathbb{C}\setminus[0,\infty)$, the [quadratic formula](/theorems/1301) gives
\begin{align*}
m_\gamma(z)=\frac{-(z+\gamma-1) \pm \sqrt{(z+\gamma-1)^2-4\gamma z}}{2\gamma z}.
\end{align*}
Since
\begin{align*}
(z+\gamma-1)^2-4\gamma z=(z-a_\gamma)(z-b_\gamma),
\end{align*}
the two possible branches are
\begin{align*}
m_\gamma(z)=\frac{-(z+\gamma-1) \pm \sqrt{(z-a_\gamma)(z-b_\gamma)}}{2\gamma z}.
\end{align*}
Let
\begin{align*}
R_\gamma: \mathbb{C}\setminus[a_\gamma,b_\gamma] &\to \mathbb{C}
\end{align*}
denote the analytic branch satisfying $R_\gamma(z)^2=(z-a_\gamma)(z-b_\gamma)$ and
\begin{align*}
\frac{R_\gamma(z)}{z}\to 1
\end{align*}
as $|z|\to\infty$ in $\mathbb{C}\setminus[a_\gamma,b_\gamma]$. Since $a_\gamma+b_\gamma=2(1+\gamma)$, the expansion of this branch at infinity is
\begin{align*}
R_\gamma(z)=z-(1+\gamma)+o(1).
\end{align*}
With the plus sign,
\begin{align*}
z\,\frac{-(z+\gamma-1)+R_\gamma(z)}{2\gamma z}=\frac{-(z+\gamma-1)+R_\gamma(z)}{2\gamma}\to -1.
\end{align*}
With the minus sign, the product is asymptotic to $-z/\gamma$, so it cannot converge to $-1$. Hence
\begin{align*}
m_\gamma(z)=\frac{-(z+\gamma-1)+R_\gamma(z)}{2\gamma z}.
\end{align*}
This selected expression is the original Stieltjes transform $m_\gamma$ on $\mathbb{C}\setminus[0,\infty)$, because the other algebraic branch violates the normalization $z m_\gamma(z)\to -1$. Therefore the assumed Stieltjes sign condition gives, for every $z\in\mathbb{C}$ with $\operatorname{Im}z>0$,
\begin{align*}
\operatorname{Im}\left(\frac{-(z+\gamma-1)+R_\gamma(z)}{2\gamma z}\right)=\operatorname{Im}m_\gamma(z)\geq 0.
\end{align*}
Thus the chosen square-root branch is the Herglotz/Stieltjes branch in the upper half-plane; the conjugate branch has the opposite boundary imaginary part on $(a_\gamma,b_\gamma)$ and is incompatible with the sign condition.
[guided]
The quadratic formula applied to
\begin{align*}
\gamma z w^2+(z+\gamma-1)w+1=0
\end{align*}
gives
\begin{align*}
m_\gamma(z)=\frac{-(z+\gamma-1) \pm \sqrt{(z+\gamma-1)^2-4\gamma z}}{2\gamma z}.
\end{align*}
The endpoints in the theorem statement enter through the discriminant. Indeed, from
\begin{align*}
a_\gamma=(1-\sqrt{\gamma})^2
\end{align*}
and
\begin{align*}
b_\gamma=(1+\sqrt{\gamma})^2,
\end{align*}
we have
\begin{align*}
a_\gamma+b_\gamma=2(1+\gamma)
\end{align*}
and
\begin{align*}
a_\gamma b_\gamma=(1-\gamma)^2.
\end{align*}
Therefore
\begin{align*}
(z-a_\gamma)(z-b_\gamma)=z^2-2(1+\gamma)z+(1-\gamma)^2=(z+\gamma-1)^2-4\gamma z.
\end{align*}
So the algebraic formula is
\begin{align*}
m_\gamma(z)=\frac{-(z+\gamma-1) \pm \sqrt{(z-a_\gamma)(z-b_\gamma)}}{2\gamma z}.
\end{align*}
Now choose the branch. Let
\begin{align*}
R_\gamma: \mathbb{C}\setminus[a_\gamma,b_\gamma] &\to \mathbb{C}
\end{align*}
be the analytic square-root branch satisfying $R_\gamma(z)^2=(z-a_\gamma)(z-b_\gamma)$ and $R_\gamma(z)/z\to 1$ as $|z|\to\infty$. This branch has the expansion
\begin{align*}
R_\gamma(z)=z-\frac{a_\gamma+b_\gamma}{2}+o(1)=z-(1+\gamma)+o(1).
\end{align*}
For the plus sign, multiplying by $z$ gives
\begin{align*}
z\,\frac{-(z+\gamma-1)+R_\gamma(z)}{2\gamma z}=\frac{-(z+\gamma-1)+R_\gamma(z)}{2\gamma}\to -1.
\end{align*}
For the minus sign, the numerator is asymptotic to $-2z$, so the product is asymptotic to $-z/\gamma$ and cannot converge to $-1$. Thus the large-$z$ normalization selects
\begin{align*}
m_\gamma(z)=\frac{-(z+\gamma-1)+R_\gamma(z)}{2\gamma z}.
\end{align*}
Why is this the Stieltjes branch rather than merely the right asymptotic algebraic branch? The selected formula agrees with the original function $m_\gamma$ on $\mathbb{C}\setminus[0,\infty)$, because the quadratic has only two roots and the other root has the wrong normalization at infinity. Hence, for every $z$ in the upper half-plane,
\begin{align*}
\operatorname{Im}\left(\frac{-(z+\gamma-1)+R_\gamma(z)}{2\gamma z}\right)=\operatorname{Im}m_\gamma(z)\geq 0.
\end{align*}
This is exactly the Herglotz/Stieltjes sign condition. The conjugate boundary branch would replace $i\sqrt{(b_\gamma-x)(x-a_\gamma)}$ by $-i\sqrt{(b_\gamma-x)(x-a_\gamma)}$ on $(a_\gamma,b_\gamma)$, producing the opposite sign for the boundary imaginary part and therefore cannot be the Stieltjes branch in the upper half-plane.
[/guided]
[/step]
[step:Compute the upper boundary value on the support interval]
Let $x \in (a_\gamma,b_\gamma)$. The upper half-plane boundary value of $R_\gamma$ at $x$ is
\begin{align*}
R_\gamma(x+i0)=i\sqrt{(b_\gamma-x)(x-a_\gamma)}.
\end{align*}
Therefore
\begin{align*}
\lim_{\eta\downarrow 0}\operatorname{Im}m_\gamma(x+i\eta)=\operatorname{Im}\left(\frac{-(x+\gamma-1)+i\sqrt{(b_\gamma-x)(x-a_\gamma)}}{2\gamma x}\right).
\end{align*}
Thus
\begin{align*}
\lim_{\eta\downarrow 0}\operatorname{Im}m_\gamma(x+i\eta)=\frac{1}{2\gamma x}\sqrt{(b_\gamma-x)(x-a_\gamma)}.
\end{align*}
If $x \in (0,\infty)\setminus[a_\gamma,b_\gamma]$, the boundary value $R_\gamma(x+i0)$ is real, and hence
\begin{align*}
\lim_{\eta\downarrow 0}\operatorname{Im}m_\gamma(x+i\eta)=0.
\end{align*}
[guided]
We now compute the boundary value of the selected square-root branch. For $x \in (a_\gamma,b_\gamma)$, the two factors $x-a_\gamma$ and $x-b_\gamma$ have opposite signs. Approaching from the upper half-plane with the branch satisfying $R_\gamma(z)/z\to 1$, the boundary value is
\begin{align*}
R_\gamma(x+i0)=i\sqrt{(b_\gamma-x)(x-a_\gamma)}.
\end{align*}
Substituting this into the selected formula for $m_\gamma$ gives
\begin{align*}
\lim_{\eta\downarrow 0}m_\gamma(x+i\eta)=\frac{-(x+\gamma-1)+i\sqrt{(b_\gamma-x)(x-a_\gamma)}}{2\gamma x}.
\end{align*}
Taking imaginary parts yields
\begin{align*}
\lim_{\eta\downarrow 0}\operatorname{Im}m_\gamma(x+i\eta)=\frac{1}{2\gamma x}\sqrt{(b_\gamma-x)(x-a_\gamma)}.
\end{align*}
This is non-negative, as required by the upper half-plane sign condition.
If $x \in (0,\infty)\setminus[a_\gamma,b_\gamma]$, then $(x-a_\gamma)(x-b_\gamma)$ is non-negative and the selected boundary value of $R_\gamma$ is real. The term $-(x+\gamma-1)$ is also real, so the boundary value of $m_\gamma$ is real. Hence
\begin{align*}
\lim_{\eta\downarrow 0}\operatorname{Im}m_\gamma(x+i\eta)=0.
\end{align*}
[/guided]
[/step]
[step:Apply Stieltjes inversion to recover the density]
Let $\mu_\gamma$ denote the probability measure on $[0,\infty)$ whose Stieltjes transform is $m_\gamma$. Let $\mathcal{L}^1$ denote one-dimensional Lebesgue measure on $\mathbb{R}$. The hypotheses give precisely the Stieltjes representation
\begin{align*}
m_\gamma(z)=\int_{[0,\infty)}\frac{1}{x-z}\,d\mu_\gamma(x),
\end{align*}
for $z \in \mathbb{C}\setminus[0,\infty)$, where $\mu_\gamma$ is finite because it is a probability measure. Hence the hypotheses of the [Stieltjes inversion theorem](/theorems/TEMP-37) apply with this sign convention, and the density of the absolutely continuous part of $\mu_\gamma$ is
\begin{align*}
f_\gamma(x)=\frac{1}{\pi}\lim_{\eta\downarrow 0}\operatorname{Im}m_\gamma(x+i\eta)
\end{align*}
for $\mathcal{L}^1$-almost every $x \in (0,\infty)$ at which the boundary limit exists. The boundary computation above gives
\begin{align*}
f_\gamma(x)=\frac{1}{2\pi\gamma x}\sqrt{(b_\gamma-x)(x-a_\gamma)}\,\mathbb{1}_{[a_\gamma,b_\gamma]}(x).
\end{align*}
Equivalently,
\begin{align*}
d\mu_{\gamma,\mathrm{ac}}(x)=f_\gamma(x)\,d\mathcal{L}^1(x).
\end{align*}
This is exactly the density claimed in the theorem statement.
[guided]
Let $\mu_\gamma$ be the probability measure on $[0,\infty)$ whose Stieltjes transform is $m_\gamma$. Let $\mathcal{L}^1$ denote one-dimensional Lebesgue measure on $\mathbb{R}$. We apply the [Stieltjes inversion theorem](/theorems/TEMP-37) with the convention
\begin{align*}
m_\gamma(z)=\int_{[0,\infty)}\frac{1}{x-z}\,d\mu_\gamma(x).
\end{align*}
The formula applies because $\mu_\gamma$ is a finite Borel measure, indeed a probability measure, and $m_\gamma$ is its Stieltjes transform on $\mathbb{C}\setminus[0,\infty)$. For this sign convention, the absolutely continuous density is recovered from upper half-plane boundary values by
\begin{align*}
f_\gamma(x)=\frac{1}{\pi}\lim_{\eta\downarrow 0}\operatorname{Im}m_\gamma(x+i\eta)
\end{align*}
for $\mathcal{L}^1$-almost every $x \in (0,\infty)$ where the limit exists.
The preceding step computed those limits explicitly. On the interval $(a_\gamma,b_\gamma)$,
\begin{align*}
\lim_{\eta\downarrow 0}\operatorname{Im}m_\gamma(x+i\eta)=\frac{1}{2\gamma x}\sqrt{(b_\gamma-x)(x-a_\gamma)}.
\end{align*}
Outside $[a_\gamma,b_\gamma]$ inside $(0,\infty)$, the boundary value is real, so the limiting imaginary part is zero. Combining these two cases gives
\begin{align*}
f_\gamma(x)=\frac{1}{2\pi\gamma x}\sqrt{(b_\gamma-x)(x-a_\gamma)}\,\mathbb{1}_{[a_\gamma,b_\gamma]}(x).
\end{align*}
Thus the absolutely continuous part satisfies
\begin{align*}
d\mu_{\gamma,\mathrm{ac}}(x)=f_\gamma(x)\,d\mathcal{L}^1(x),
\end{align*}
which is the asserted Marchenko-Pastur density.
[/guided]
[/step]