[proofplan] We prove both directions of both equivalences simultaneously. Let $f = \mathbb{P}_i(T_i^+ < \infty)$ be the return probability and $V_i = \sum_{n=0}^{\infty} \mathbb{1}_{X_n = i}$ the total number of visits to $i$. Using the [Strong Markov Property](/theorems/2208) applied iteratively at successive return times, we show $\mathbb{P}_i(V_i \geq k) = f^{k-1}$ for all $k \geq 1$. When $f = 1$ (recurrent), this gives $\mathbb{P}_i(V_i = \infty) = 1$, so $i$ is visited infinitely often a.s. When $f < 1$ (transient), $\mathbb{P}_i(V_i \geq k) \to 0$, so $\mathbb{P}_i(V_i = \infty) = 0$, and there are no intermediate cases. [/proofplan] [step:Define the total visit count and successive return times] Let $X_0 = i$. Define the total number of visits to state $i$ (including the initial visit at time $0$): \begin{align*} V_i = \sum_{n=0}^{\infty} \mathbb{1}_{X_n = i} = |\{n \geq 0 : X_n = i\}|. \end{align*} Define the successive return times to $i$ inductively: let $T_i^{(1)} = T_i^+ = \inf\{n \geq 1 : X_n = i\}$ be the first return time, and for $k \geq 2$, \begin{align*} T_i^{(k)} = \inf\{n > T_i^{(k-1)} : X_n = i\}, \end{align*} with $T_i^{(k)} = \infty$ if $T_i^{(k-1)} = \infty$ or no further return occurs. Each $T_i^{(k)}$ is a stopping time. Write $f = \mathbb{P}_i(T_i^+ < \infty)$ for the return probability. [/step] [step:Apply the strong Markov property to show $\mathbb{P}_i(V_i \geq k) = f^{k-1}$] We prove by induction on $k$ that $\mathbb{P}_i(V_i \geq k) = f^{k-1}$ for all $k \geq 1$. **Base case** ($k = 1$): Since $X_0 = i$, we have $V_i \geq 1$ with probability one. Thus $\mathbb{P}_i(V_i \geq 1) = 1 = f^0$. **Inductive step**: Suppose $\mathbb{P}_i(V_i \geq k) = f^{k-1}$. The event $\{V_i \geq k+1\}$ requires that the chain returns to $i$ at least $k$ times after time $0$, i.e., $T_i^{(k)} < \infty$. We decompose: \begin{align*} \mathbb{P}_i(V_i \geq k+1) = \mathbb{P}_i(T_i^{(k)} < \infty \text{ and the chain returns to } i \text{ after } T_i^{(k)}). \end{align*} By the [Strong Markov Property](/theorems/2208) applied at the stopping time $T_i^{(k)}$: on the event $\{T_i^{(k)} < \infty\}$, we have $X_{T_i^{(k)}} = i$, and the post-$T_i^{(k)}$ process $(X_{T_i^{(k)}+n})_{n \geq 0}$ is a Markov chain started at $i$ with the same transition matrix $P$, independent of $(X_0, \ldots, X_{T_i^{(k)}})$. Therefore the probability of a further return to $i$ after $T_i^{(k)}$ is exactly $f$, independently of the past. Hence: \begin{align*} \mathbb{P}_i(V_i \geq k+1) = \mathbb{P}_i(T_i^{(k)} < \infty) \cdot f = f^{k-1} \cdot f = f^k. \end{align*} This completes the induction: $\mathbb{P}_i(V_i \geq k) = f^{k-1}$ for all $k \geq 1$. [guided] The strong Markov property is the essential tool here. At each successive return time $T_i^{(k)}$, the chain arrives at state $i$ and "starts fresh": its future evolution is an independent copy of a Markov chain from $i$. This means that each inter-return attempt — the question "will the chain return to $i$ yet again?" — is an independent Bernoulli trial with success probability $f$. Why does the strong Markov property apply? We need to verify that $T_i^{(k)}$ is a stopping time. This holds because $\{T_i^{(k)} = n\}$ is determined by $(X_0, X_1, \ldots, X_n)$: it requires that $X_n = i$ and exactly $k - 1$ of $X_1, \ldots, X_{n-1}$ equal $i$, which depends only on the trajectory up to time $n$. The induction gives $\mathbb{P}_i(V_i \geq k) = f^{k-1}$. Equivalently, $\mathbb{P}_i(T_i^{(k)} < \infty) = f^{k-1}$, since $V_i \geq k$ if and only if the chain makes at least $k-1$ returns after time $0$. The multiplicative structure $f^{k-1}$ reflects the independence: the $k-1$ successive return attempts each succeed with probability $f$, independently of one another. [/guided] [/step] [step:Conclude the recurrent case: $f = 1$ implies infinitely many visits a.s.] Suppose $i$ is recurrent, so $f = \mathbb{P}_i(T_i^+ < \infty) = 1$. Then for every $k \geq 1$: \begin{align*} \mathbb{P}_i(V_i \geq k) = f^{k-1} = 1. \end{align*} The events $\{V_i \geq k\}$ form a decreasing sequence with intersection $\{V_i = \infty\}$. By continuity of probability from above: \begin{align*} \mathbb{P}_i(V_i = \infty) = \lim_{k \to \infty} \mathbb{P}_i(V_i \geq k) = \lim_{k \to \infty} 1 = 1. \end{align*} Since $\{V_i = \infty\} = \{X_n = i \text{ for infinitely many } n\}$, this gives $\mathbb{P}_i(X_n = i \text{ for infinitely many } n) = 1$. [/step] [step:Conclude the transient case: $f < 1$ implies finitely many visits a.s.] Suppose $i$ is transient, so $f = \mathbb{P}_i(T_i^+ < \infty) < 1$. Then for every $k \geq 1$: \begin{align*} \mathbb{P}_i(V_i \geq k) = f^{k-1} \to 0 \quad \text{as } k \to \infty. \end{align*} By the same continuity argument: \begin{align*} \mathbb{P}_i(V_i = \infty) = \lim_{k \to \infty} f^{k-1} = 0. \end{align*} Hence $\mathbb{P}_i(X_n = i \text{ for infinitely many } n) = 0$. Moreover, since $\mathbb{P}_i(V_i \geq k) = f^{k-1}$, the number of returns $V_i - 1$ (visits after time $0$) has a geometric distribution: $\mathbb{P}_i(V_i - 1 = r) = f^r(1 - f)$ for $r \geq 0$. The expected total number of visits is: \begin{align*} \mathbb{E}_i[V_i] = \sum_{k=1}^{\infty} \mathbb{P}_i(V_i \geq k) = \sum_{k=1}^{\infty} f^{k-1} = \frac{1}{1 - f} < \infty. \end{align*} [guided] The dichotomy is complete and sharp: there is no middle ground between "infinitely many visits with probability one" and "finitely many visits with probability one". This follows from the geometric structure $\mathbb{P}_i(V_i \geq k) = f^{k-1}$: the only values of $f \in [0, 1]$ for which $\lim_{k \to \infty} f^{k-1}$ is non-zero are $f = 1$ (giving limit $1$) and $f < 1$ (giving limit $0$). There is no $f$ for which the chain visits $i$ infinitely often with some intermediate probability $p \in (0, 1)$. The expected number of visits $\mathbb{E}_i[V_i] = 1/(1-f)$ connects to the [Recurrence Summability Criterion](/theorems/2211): by linearity of expectation, \begin{align*} \mathbb{E}_i[V_i] = \sum_{n=0}^{\infty} \mathbb{P}_i(X_n = i) = \sum_{n=0}^{\infty} p_{ii}(n). \end{align*} So $\sum_n p_{ii}(n) = 1/(1-f)$, which is infinite if and only if $f = 1$, recovering the summability criterion. [/guided] [/step]