[proofplan]
We prove that every proper holomorphic self-map $f: B^n \to B^n$ with $n \geq 2$ is an automorphism. The proof proceeds in three stages: (1) by Fefferman's [boundary regularity](/theorems/99) theorem, $f$ extends smoothly to $\overline{B^n}$ and maps $\partial B^n$ to $\partial B^n$; (2) by a differential-geometric argument, the boundary map $f|_{\partial B^n}: \partial B^n \to \partial B^n$ is a CR diffeomorphism, and the rigidity of CR automorphisms of the sphere $S^{2n-1}$ forces $f|_{\partial B^n}$ to be the restriction of a Mobius transformation of $B^n$; (3) by the [identity principle](/theorems/3357), $f$ agrees with this automorphism on all of $B^n$.
[/proofplan]
[step:Apply Fefferman's theorem to extend $f$ smoothly to $\overline{B^n}$]
Let $f: B^n \to B^n$ be a proper holomorphic map, where $B^n = \{z \in \mathbb{C}^n : |z| < 1\}$ is the open unit ball. Properness means that for every compact set $K \subset B^n$, the preimage $f^{-1}(K)$ is compact in $B^n$, or equivalently, $|f(z)| \to 1$ as $|z| \to 1$.
Since $B^n$ has smooth, strictly pseudoconvex boundary (the defining function $\rho(z) = |z|^2 - 1$ has Levi form equal to the identity on the complex tangent space of $\partial B^n$), Fefferman's theorem on [boundary regularity](/theorems/99) of biholomorphic maps between strictly pseudoconvex domains applies. In its extended form for proper maps, the theorem states: if $f: \Omega_1 \to \Omega_2$ is a proper holomorphic map between bounded domains with smooth strictly pseudoconvex boundaries, then $f$ extends to a $C^\infty$ map $f: \overline{\Omega_1} \to \overline{\Omega_2}$.
We verify the hypotheses of Fefferman's theorem. The domain $B^n$ is bounded and has smooth boundary $\partial B^n = S^{2n-1}$, the unit sphere in $\mathbb{C}^n$. The boundary is strictly pseudoconvex: for $\rho(z) = |z|^2 - 1$, the Levi form is
\begin{align*}
\mathcal{L}_\rho(z; w) = \sum_{j,k=1}^n \frac{\partial^2 \rho}{\partial z_j \partial \bar{z}_k} w_j \bar{w}_k = \sum_{j=1}^n |w_j|^2 = |w|^2 > 0
\end{align*}
for all $w \neq 0$, so $\partial B^n$ is strictly pseudoconvex. Both the source and target are $B^n$, so both boundaries satisfy the hypotheses. Therefore $f$ extends to a smooth map $f: \overline{B^n} \to \overline{B^n}$.
Since $f$ is proper, $f(\partial B^n) \subset \partial B^n$: if $z_k \to z_0 \in \partial B^n$, then $|z_k| \to 1$, so by properness $|f(z_k)| \to 1$, hence $f(z_0) \in \partial B^n$ by continuity of the extension.
[guided]
Fefferman's original 1974 theorem proved that a biholomorphic map between smoothly bounded strictly pseudoconvex domains extends smoothly to the boundary. The extension to proper holomorphic maps (not necessarily bijective) was established by Bell and Ligocka, and independently by Bell using the theory of the Bergman projection. The key ingredients are:
1. The Bergman projection $P: L^2(B^n) \to \mathcal{O}(B^n) \cap L^2(B^n)$ satisfies a regularity property called "Condition R": $P$ maps $C^\infty(\overline{B^n})$ to $C^\infty(\overline{B^n})$. This holds for strictly pseudoconvex domains by Kohn's solution of the $\bar\partial$-Neumann problem.
2. A proper holomorphic map satisfies Bell's transformation formula: if $f: \Omega_1 \to \Omega_2$ is proper holomorphic, then $f^* \circ P_2 = P_1 \circ f^*$ up to a Jacobian factor (where $P_1, P_2$ are the Bergman projections). Combined with Condition R, this forces $f$ to be smooth up to the boundary.
Why does properness force $f(\partial B^n) \subset \partial B^n$? By definition, $f$ is proper if and only if $f^{-1}(K)$ is compact for every compact $K \subset B^n$. Equivalently, $|f(z)| \to 1$ whenever $|z| \to 1$ (since the only way for preimages of compact sets to escape compactness is to approach the boundary). Once $f$ extends continuously to $\overline{B^n}$, this forces $|f(z)| = 1$ for $z \in \partial B^n$.
[/guided]
[/step]
[step:Show that $f|_{\partial B^n}$ is a CR map of the sphere and determine the branch locus]
The smooth extension $f: \overline{B^n} \to \overline{B^n}$ maps $\partial B^n$ to $\partial B^n$. The restriction $f|_{\partial B^n}: S^{2n-1} \to S^{2n-1}$ is a CR map: it maps the CR structure of the source sphere to the CR structure of the target sphere. This holds because $f$ is holomorphic on $B^n$ and smooth up to the boundary, so the tangential Cauchy--Riemann equations are preserved in the limit.
More precisely, the CR structure on $S^{2n-1}$ is defined by the complex tangent space $T^{1,0}_z(\partial B^n) = T^{1,0}_z \mathbb{C}^n \cap (T_z(\partial B^n) \otimes \mathbb{C})$, which has complex dimension $n-1$. A smooth map $g: S^{2n-1} \to S^{2n-1}$ is CR if $dg_z(T^{1,0}_z(\partial B^n)) \subset T^{1,0}_{g(z)}(\partial B^n)$ for all $z \in \partial B^n$. Since $f$ is holomorphic in $B^n$, the differential $df_z$ maps $(1,0)$-vectors to $(1,0)$-vectors for $z \in B^n$, and by smooth extension to the boundary, this property persists for $z \in \partial B^n$.
We now analyse the branch locus. Define the branch locus of $f$ as
\begin{align*}
V_f := \{z \in B^n : \det Jf_z = 0\},
\end{align*}
where $Jf_z$ is the $n \times n$ Jacobian matrix with entries $(Jf_z)_{jk} = \partial f_j / \partial z_k(z)$. Since $f$ is holomorphic and non-constant, $\det Jf_z$ is a [holomorphic function](/page/Holomorphic%20Function) that is not identically zero (otherwise $f$ would map $B^n$ into a lower-dimensional set, contradicting properness). By the [Identity Principle](/theorems/3357), $V_f$ is a complex-analytic variety of codimension at least one in $B^n$.
[/step]
[step:Apply the CR rigidity of $S^{2n-1}$ to conclude that $f|_{\partial B^n}$ is a Mobius transformation]
The sphere $S^{2n-1}$ for $n \geq 2$ is a model CR manifold with a rigid CR automorphism group. The CR automorphisms of $S^{2n-1}$ are precisely the restrictions to $\partial B^n$ of the holomorphic automorphisms of $B^n$, which form the group $\operatorname{Aut}(B^n)$. Each automorphism of $B^n$ is a Mobius transformation of the form
\begin{align*}
\phi_a: B^n &\to B^n, \\
z &\mapsto \frac{a - P_a(z) - s_a Q_a(z)}{1 - \langle z, a \rangle},
\end{align*}
composed with a unitary rotation $U \in U(n)$, where $a \in B^n$, $P_a$ is the [orthogonal projection](/theorems/437) onto $\mathbb{C} \cdot a$, $Q_a = I - P_a$, $s_a = (1 - |a|^2)^{1/2}$, and $\langle z, a \rangle = \sum_{j=1}^n z_j \bar{a}_j$.
The key rigidity result is: for $n \geq 2$, every smooth CR map $g: S^{2n-1} \to S^{2n-1}$ that is a local diffeomorphism at some point is the restriction of an automorphism of $B^n$. This is a theorem of Pinchuk (building on work of Tanaka and Chern--Moser on the CR geometry of strictly pseudoconvex hypersurfaces).
We verify the hypothesis: $f|_{\partial B^n}$ is a local diffeomorphism at some point of $\partial B^n$. Since $f: B^n \to B^n$ is proper, the generic fibre $f^{-1}(w)$ has constant cardinality $m$ (counted with multiplicity) for $w$ in the complement of the critical values. The branch locus $V_f$ is a proper analytic subvariety of $B^n$, so $V_f \cap \partial B^n$ has measure zero in $\partial B^n$ (since $V_f$ has real codimension at least two). At any point $z_0 \in \partial B^n$ where $\det Jf_{z_0} \neq 0$, the map $f|_{\partial B^n}$ is a local diffeomorphism by the [inverse function theorem](/page/Inverse%20Function%20Theorem). Such points exist because $V_f \cap \partial B^n$ has measure zero in $\partial B^n$.
Therefore, by Pinchuk's theorem, $f|_{\partial B^n}$ is the restriction of some $\psi \in \operatorname{Aut}(B^n)$.
[guided]
This is the step where the hypothesis $n \geq 2$ is essential. In one complex variable ($n = 1$), the boundary of the unit disc $\partial\mathbb{D} = S^1$ has no CR structure (it is one-dimensional real), and there exist proper holomorphic self-maps of $\mathbb{D}$ that are not automorphisms -- for example, the Blaschke products $z \mapsto z^m$ for $m \geq 2$.
For $n \geq 2$, the sphere $S^{2n-1}$ has a rich CR structure (the complex tangent space has dimension $n - 1 \geq 1$), and this structure is "rigid" in the sense that any CR map preserving it is very constrained. The precise rigidity theorem we use (Pinchuk's theorem) states that a smooth CR map $g: S^{2n-1} \to S^{2n-1}$ that is a local CR diffeomorphism at some point extends to a holomorphic automorphism of $B^n$.
Why does CR rigidity hold for $n \geq 2$? The underlying reason is the non-degeneracy of the Levi form of $S^{2n-1}$. The Levi form has $n - 1$ positive eigenvalues, and the Chern--Moser theory shows that the local CR invariants of a strictly pseudoconvex hypersurface are captured by a finite number of geometric quantities. For the sphere, which is the "flat model" (all CR invariants vanish), any local CR equivalence with the sphere must be globally a Mobius transformation. This is analogous to the fact that a local isometry of $S^n$ (with the round metric) extends to a global isometry.
Why does the branch locus $V_f$ not prevent us from finding a point where $f|_{\partial B^n}$ is a local diffeomorphism? The branch locus is a proper complex-analytic subvariety of $B^n$, so it has real codimension at least $2$ in $B^n$ (complex codimension at least $1$ means real codimension at least $2$). Its closure in $\overline{B^n}$ intersects $\partial B^n$ in a set of real codimension at least $1$ in $\partial B^n$ (at least), and in particular has zero $(2n-1)$-dimensional measure in $\partial B^n$. So generic boundary points are non-branch points.
[/guided]
[/step]
[step:Conclude by the identity principle that $f = \psi$ on all of $B^n$]
We have established that $f|_{\partial B^n} = \psi|_{\partial B^n}$ for some automorphism $\psi \in \operatorname{Aut}(B^n)$. Consider the holomorphic map
\begin{align*}
g := \psi^{-1} \circ f: B^n \to B^n.
\end{align*}
The map $g$ is holomorphic on $B^n$, extends smoothly to $\overline{B^n}$ (since both $\psi^{-1}$ and $f$ do), and satisfies $g|_{\partial B^n} = \operatorname{id}|_{\partial B^n}$, i.e., $g(z) = z$ for all $z \in \partial B^n$.
We claim $g = \operatorname{id}$ on all of $B^n$. Consider the holomorphic map $h := g - \operatorname{id}: \overline{B^n} \to \mathbb{C}^n$, whose components $h_j = g_j - z_j$ are holomorphic on $B^n$ and continuous on $\overline{B^n}$, with $h_j|_{\partial B^n} = 0$ for $j = 1, \ldots, n$. Each $h_j$ is a [holomorphic function](/page/Holomorphic%20Function) on the connected [open set](/page/Open%20Set) $B^n$ that vanishes on $\partial B^n$. Since $\partial B^n$ contains an open subset of a totally real submanifold of maximal dimension (in fact, $\partial B^n$ is a real hypersurface, and the zero set of a non-trivial [holomorphic function](/page/Holomorphic%20Function) on $B^n$ that is continuous up to $\overline{B^n}$ cannot contain an open subset of $\partial B^n$), we conclude $h_j \equiv 0$ on $B^n$.
Alternatively, by the [maximum modulus principle](/page/Maximum%20Modulus%20Principle) applied to each component: $|h_j(z)| \leq \max_{\partial B^n} |h_j| = 0$, so $h_j \equiv 0$ on $B^n$.
Therefore $g = \operatorname{id}$, which gives $f = \psi \in \operatorname{Aut}(B^n)$.
[guided]
The final step is where all the pieces come together. We have shown:
1. $f$ extends smoothly to $\overline{B^n}$ (Fefferman).
2. $f|_{\partial B^n}$ is a CR map of $S^{2n-1}$ (because $f$ is holomorphic).
3. $f|_{\partial B^n}$ equals $\psi|_{\partial B^n}$ for some automorphism $\psi$ (CR rigidity, Pinchuk).
The composition $g = \psi^{-1} \circ f$ is a proper holomorphic self-map of $B^n$ that restricts to the identity on the boundary. The question is: can a [holomorphic function](/page/Holomorphic%20Function) on $B^n$ that is the identity on $\partial B^n$ be anything other than the identity on $B^n$?
The answer is no, by the [maximum modulus principle](/theorems/491). For each component $h_j = g_j - z_j$, we have $h_j$ holomorphic on $B^n$, continuous on $\overline{B^n}$, and $h_j|_{\partial B^n} = 0$. The [maximum modulus principle](/page/Maximum%20Modulus%20Principle) states that $\max_{\overline{B^n}} |h_j| = \max_{\partial B^n} |h_j| = 0$, so $h_j \equiv 0$ on $B^n$.
This completes the proof that $f = \psi$ is an automorphism of $B^n$. The result is sharp in the following sense: for $n = 1$, proper holomorphic self-maps of $\mathbb{D}$ include Blaschke products of arbitrary degree, so the conclusion fails. The rigidity of the CR structure in $n \geq 2$ is what distinguishes the two cases.
[/guided]
[/step]