[proofplan]
We first show that the extinction probability $q = \lim_{n \to \infty} \mathbb{P}(Z_n = 0)$ satisfies the fixed-point equation $F(q) = q$. Then we prove $q$ is the smallest non-negative root by showing that every fixed point in $[0,1]$ is at least $q$. For part (ii), we analyse the convexity of $F$ on $[0,1]$: when $\mu \le 1$, the graph of $F$ lies above the diagonal on $[0,1)$, forcing the smallest fixed point to be $1$; when $\mu > 1$, the slope $F'(1) = \mu > 1$ means $F$ crosses the diagonal at some $q < 1$.
[/proofplan]
[step:Show that $q$ satisfies the fixed-point equation $F(q) = q$]
Define $q_n = \mathbb{P}(Z_n = 0)$. Since $\{Z_n = 0\} \subset \{Z_{n+1} = 0\}$ (once extinct, always extinct), the sequence $(q_n)$ is non-decreasing and bounded above by $1$, so $q = \lim_{n \to \infty} q_n$ exists in $[0, 1]$.
By the iteration formula $F_n(z) = F(F_{n-1}(z))$ applied at $z = 0$,
\begin{align*}
q_{n+1} = \mathbb{P}(Z_{n+1} = 0) = F_{n+1}(0) = F(F_n(0)) = F(q_n).
\end{align*}
Since $F$ is continuous on $[0, 1]$ (as a [power series](/page/Power%20Series) converging on $[0, 1]$), taking $n \to \infty$ gives
\begin{align*}
q = \lim_{n \to \infty} q_{n+1} = \lim_{n \to \infty} F(q_n) = F\!\left(\lim_{n \to \infty} q_n\right) = F(q).
\end{align*}
[/step]
[step:Prove that $q$ is the smallest non-negative fixed point]
Let $s \in [0, 1]$ satisfy $F(s) = s$. We show $q_n \le s$ for all $n$ by induction, which gives $q \le s$.
**Base case.** $q_0 = \mathbb{P}(Z_0 = 0) = 0 \le s$.
**Inductive step.** Assume $q_n \le s$. Since $F$ is a power series with non-negative coefficients, $F$ is non-decreasing on $[0, 1]$. Therefore
\begin{align*}
q_{n+1} = F(q_n) \le F(s) = s.
\end{align*}
By induction, $q_n \le s$ for all $n$, so $q = \lim_{n \to \infty} q_n \le s$. Since $s$ was an arbitrary fixed point in $[0, 1]$, the extinction probability $q$ is the smallest such fixed point.
[/step]
[step:Analyse the case $\mu \le 1$: certain extinction]
We exclude the degenerate case $\mathbb{P}(Z_1 = 1) = 1$ (deterministic population, $\mu = 1$, $q = 1$). In all other cases with $\mu \le 1$, we show $q = 1$ by proving that $s = 1$ is the only fixed point of $F$ in $[0, 1]$.
The offspring pgf $F(z) = \sum_{k=0}^{\infty} p_k z^k$ has the following properties on $[0, 1]$:
- $F(1) = 1$ (probabilities sum to $1$).
- $F'(z) = \sum_{k=1}^{\infty} k p_k z^{k-1} \ge 0$, so $F$ is non-decreasing.
- $F''(z) = \sum_{k=2}^{\infty} k(k-1) p_k z^{k-2} \ge 0$, so $F$ is convex on $[0, 1]$.
- $F'(1^-) = \mu \le 1$.
[guided]
Why does convexity decide the question? A convex function $F$ on $[0,1]$ with $F(1) = 1$ lies entirely above its tangent line at $z = 1$. That tangent line has slope $F'(1^-) = \mu$ and passes through $(1, 1)$, so it is $\ell(z) = 1 + \mu(z - 1)$. Convexity gives $F(z) \ge \ell(z)$ for all $z \in [0, 1]$.
When $\mu \le 1$, the tangent line satisfies $\ell(z) = 1 - \mu(1-z) \ge 1 - (1-z) = z$ for $z \in [0, 1]$ (using $\mu \le 1$). Therefore $F(z) \ge z$ for all $z \in [0, 1]$, with equality only possible at $z = 1$ (since $F$ is strictly convex unless all offspring mass is on a single value — but if $\mathbb{P}(Z_1 = 1) < 1$ and $\mu \le 1$, strict convexity on $(0, 1)$ prevents equality before $z = 1$).
More precisely: if $F(s) = s$ for some $s \in [0, 1)$, then by convexity $F$ lies below the secant line from $(s, s)$ to $(1, 1)$, which is the diagonal $y = z$. But we have just shown $F(z) \ge z$. So $F(z) = z$ on $[s, 1]$, forcing $F$ to be linear on $[s, 1]$ — contradicting strict convexity unless the offspring distribution is degenerate.
[/guided]
Since $F$ is convex and $F(1) = 1$, for any $z \in [0, 1)$,
\begin{align*}
F(z) \ge F(1) + F'(1^-)(z - 1) = 1 + \mu(z - 1) = 1 - \mu(1 - z).
\end{align*}
When $\mu \le 1$, this gives $F(z) \ge 1 - (1 - z) = z$. If $F(s) = s$ for some $s \in [0, 1)$, then $F(z) = z$ throughout $[s, 1]$ by convexity (a convex function that touches a linear function from above at two points must equal it in between). This forces $F$ to be linear, meaning $\mathbb{P}(Z_1 = 1) = 1$, which we excluded. Therefore $s = 1$ is the only fixed point, and $q = 1$.
[/step]
[step:Analyse the case $\mu > 1$: survival with positive probability]
When $\mu > 1$, we have $F'(1^-) = \mu > 1$. Since $F(0) = p_0 \ge 0$ and $F(1) = 1$, and $F$ is continuous on $[0, 1]$, consider the function $h(z) = F(z) - z$. We have:
- $h(0) = F(0) = p_0 \ge 0$.
- $h(1) = F(1) - 1 = 0$.
- $h'(1^-) = F'(1^-) - 1 = \mu - 1 > 0$.
Since $h'(1^-) > 0$, the function $h$ is strictly increasing near $z = 1$, so $h(z) < h(1) = 0$ for $z$ slightly less than $1$. Since $h(0) \ge 0$ and $h$ is continuous, by the intermediate value theorem there exists $s \in [0, 1)$ with $h(s) = 0$, i.e., $F(s) = s$.
By convexity of $F$ on $[0, 1]$ and the fact that $F$ crosses the diagonal at $s < 1$ and meets it again at $1$, the smallest such $s$ is the extinction probability $q$, and $q < 1$. The population survives with probability $1 - q > 0$.
[/step]
