[proofplan] We write the deleted normal matrix as a rank-one perturbation of the full normal matrix. The key algebraic input is the Sherman–Morrison inverse identity, which we verify directly for the matrix $X^\top X-x_i x_i^\top$. Substituting this inverse into the deleted normal equations gives the leave-one-out correction for $\hat{\beta}_{(i)}$. Multiplying by $x_i^\top$ then converts the coefficient formula into the deleted residual formula. [/proofplan]

[step:Express the deleted normal equations as a rank-one update] Define \begin{align*} A := X^\top X \in \mathbb{R}^{p\times p}, \qquad C := A^{-1} \in \mathbb{R}^{p\times p}, \qquad x := x_i \in \mathbb{R}^p, \qquad s := y_i \in \mathbb{R}. \end{align*} Since $X$ has full column rank, $A$ is invertible. By deleting the $i$-th row of $X$ and the $i$-th component of $y$, we have \begin{align*} X_{(i)}^\top X_{(i)} = A - x x^\top, \qquad X_{(i)}^\top y_{(i)} = X^\top y - x s. \end{align*} Also, \begin{align*} h_{ii} = x^\top Cx, \qquad \hat{\beta}=CX^\top y, \qquad e_i=s-x^\top \hat{\beta}. \end{align*} [/step]

[step:Invert the deleted normal matrix by verifying the Sherman–Morrison identity]We claim that \begin{align*} (A-xx^\top)^{-1} = C+\frac{Cxx^\top C}{1-x^\top Cx}. \end{align*} The denominator is nonzero because $1-x^\top Cx=1-h_{ii}>0$. Define \begin{align*} M := C+\frac{Cxx^\top C}{1-x^\top Cx} \in \mathbb{R}^{p\times p}, \qquad I_p := \operatorname{id}_{\mathbb{R}^p} \in \mathbb{R}^{p\times p}. \end{align*} Here $I_p$ denotes the $p \times p$ identity matrix. Then \begin{align*} (A-xx^\top)M &= AC + \frac{ACxx^\top C}{1-x^\top Cx} - xx^\top C - \frac{xx^\top Cxx^\top C}{1-x^\top Cx} \\ &= I_p + \frac{xx^\top C}{1-x^\top Cx} - xx^\top C - \frac{x(x^\top Cx)x^\top C}{1-x^\top Cx} \\ &= I_p + \frac{xx^\top C-(1-x^\top Cx)xx^\top C-(x^\top Cx)xx^\top C}{1-x^\top Cx} \\ &= I_p. \end{align*} Thus $A-xx^\top$ has a right inverse $M$. Since it is a square matrix, this right inverse is its inverse, so $X_{(i)}^\top X_{(i)}=A-xx^\top$ is invertible and the displayed formula holds.[/step]

[guided]The deleted normal matrix differs from the full normal matrix by removing exactly the contribution of the deleted row. Since the contribution of row $i$ to $X^\top X$ is $x_i x_i^\top$, this is a rank-one perturbation. We now verify the inverse formula directly, rather than treating it as a black box. Let \begin{align*} M := C+\frac{Cxx^\top C}{1-x^\top Cx} \in \mathbb{R}^{p\times p}, \qquad I_p := \operatorname{id}_{\mathbb{R}^p} \in \mathbb{R}^{p\times p}. \end{align*} Here $I_p$ denotes the $p \times p$ identity matrix. This matrix $M$ is well-defined because $1-x^\top Cx=1-h_{ii}>0$. We multiply $M$ by $A-xx^\top$ on the left: \begin{align*} (A-xx^\top)M &= (A-xx^\top)\left(C+\frac{Cxx^\top C}{1-x^\top Cx}\right) \\ &= AC + \frac{ACxx^\top C}{1-x^\top Cx} - xx^\top C - \frac{xx^\top Cxx^\top C}{1-x^\top Cx}. \end{align*} Because $C=A^{-1}$, we have $AC=I_p$ and $ACx=x$. Also $x^\top Cx$ is a scalar, so \begin{align*} xx^\top Cxx^\top C = x(x^\top Cx)x^\top C = (x^\top Cx)xx^\top C. \end{align*} Substituting these identities gives \begin{align*} (A-xx^\top)M &= I_p + \frac{xx^\top C}{1-x^\top Cx} - xx^\top C - \frac{(x^\top Cx)xx^\top C}{1-x^\top Cx} \\ &= I_p + \frac{xx^\top C-(1-x^\top Cx)xx^\top C-(x^\top Cx)xx^\top C}{1-x^\top Cx} \\ &= I_p. \end{align*} Therefore $M$ is a right inverse of the square matrix $A-xx^\top$. A square matrix with a right inverse is invertible, and the right inverse equals the inverse. Hence \begin{align*} (A-xx^\top)^{-1} = C+\frac{Cxx^\top C}{1-x^\top Cx}. \end{align*}[/guided]

[step:Substitute the inverse formula into the deleted estimator]Using the deleted normal equations and the inverse formula just proved, \begin{align*} \hat{\beta}_{(i)} &= (A-xx^\top)^{-1}(X^\top y-xs) \\ &= \left(C+\frac{Cxx^\top C}{1-x^\top Cx}\right)(X^\top y-xs) \\ &= CX^\top y-Cxs + \frac{Cx\,x^\top CX^\top y}{1-x^\top Cx} - \frac{Cx\,x^\top Cx\,s}{1-x^\top Cx}. \end{align*} Since $CX^\top y=\hat{\beta}$, $x^\top C X^\top y=x^\top \hat{\beta}$, and $x^\top Cx=h_{ii}$, this becomes \begin{align*} \hat{\beta}_{(i)} &= \hat{\beta} - Cxs + \frac{Cx\,x^\top \hat{\beta}}{1-h_{ii}} - \frac{Cx\,h_{ii}s}{1-h_{ii}} \\ &= \hat{\beta} + Cx\left( -s+\frac{x^\top\hat{\beta}-h_{ii}s}{1-h_{ii}} \right) \\ &= \hat{\beta} + Cx\left( \frac{-s(1-h_{ii})+x^\top\hat{\beta}-h_{ii}s}{1-h_{ii}} \right) \\ &= \hat{\beta} - \frac{Cx(s-x^\top\hat{\beta})}{1-h_{ii}} \\ &= \hat{\beta} - \frac{(X^\top X)^{-1}x_i e_i}{1-h_{ii}}. \end{align*}[/step]

[guided]Now we insert the inverse of the deleted normal matrix into the formula for the deleted least-squares estimator. The deleted estimator is \begin{align*} \hat{\beta}_{(i)} = (X_{(i)}^\top X_{(i)})^{-1}X_{(i)}^\top y_{(i)}. \end{align*} Using \begin{align*} X_{(i)}^\top X_{(i)}=A-xx^\top, \qquad X_{(i)}^\top y_{(i)}=X^\top y-xs, \end{align*} and the inverse identity from the previous step, we obtain \begin{align*} \hat{\beta}_{(i)} &= \left(C+\frac{Cxx^\top C}{1-x^\top Cx}\right)(X^\top y-xs) \\ &= CX^\top y-Cxs + \frac{Cx\,x^\top CX^\top y}{1-x^\top Cx} - \frac{Cx\,x^\top Cx\,s}{1-x^\top Cx}. \end{align*} Each term now has a regression interpretation. Since $CX^\top y=\hat{\beta}$, the scalar $x^\top C X^\top y$ equals $x^\top\hat{\beta}$, and $x^\top Cx=h_{ii}$. Therefore \begin{align*} \hat{\beta}_{(i)} &= \hat{\beta} - Cxs + \frac{Cx\,x^\top \hat{\beta}}{1-h_{ii}} - \frac{Cx\,h_{ii}s}{1-h_{ii}}. \end{align*} We collect the three terms containing $Cx$: \begin{align*} \hat{\beta}_{(i)} &= \hat{\beta} + Cx\left( -s+\frac{x^\top\hat{\beta}-h_{ii}s}{1-h_{ii}} \right) \\ &= \hat{\beta} + Cx\left( \frac{-s(1-h_{ii})+x^\top\hat{\beta}-h_{ii}s}{1-h_{ii}} \right) \\ &= \hat{\beta} + Cx\left( \frac{x^\top\hat{\beta}-s}{1-h_{ii}} \right). \end{align*} Because $e_i=s-x^\top\hat{\beta}$, the numerator $x^\top\hat{\beta}-s$ is $-e_i$. Returning to the original notation $C=(X^\top X)^{-1}$ and $x=x_i$, we get \begin{align*} \hat{\beta}_{(i)} = \hat{\beta} - \frac{(X^\top X)^{-1}x_i e_i}{1-h_{ii}}. \end{align*}[/guided]

[step:Evaluate the deleted residual from the coefficient formula] By definition, \begin{align*} e_{i(i)} = y_i-x_i^\top\hat{\beta}_{(i)}. \end{align*} Using the coefficient formula, \begin{align*} x_i^\top\hat{\beta}_{(i)} &= x_i^\top\hat{\beta} - \frac{x_i^\top (X^\top X)^{-1}x_i\, e_i}{1-h_{ii}} \\ &= x_i^\top\hat{\beta} - \frac{h_{ii}e_i}{1-h_{ii}}. \end{align*} Hence \begin{align*} e_{i(i)} &= y_i-x_i^\top\hat{\beta} + \frac{h_{ii}e_i}{1-h_{ii}} \\ &= e_i+\frac{h_{ii}e_i}{1-h_{ii}} \\ &= \frac{e_i}{1-h_{ii}}. \end{align*} This is the claimed deleted residual formula. [/step]