[proofplan]
The Bernoulli hypothesis makes $X_1, X_2, \ldots$ independent and identically distributed, so the log-likelihood decomposes as a sum of i.i.d. terms $-\log_2 p(X_i)$ with mean $H(X_1)$. The weak law of large numbers applied to this sum gives convergence in probability of the empirical log-likelihood $-\frac{1}{n} \log_2 p(X_1, \ldots, X_n)$ to $H(X_1)$, which is precisely the AEP with constant $H(X_1)$.
[/proofplan]
[step:Decompose the joint log-likelihood into a sum of i.i.d. random variables]
Since $X_1, X_2, \ldots$ is Bernoulli (i.i.d. with common distribution equal to that of $X_1$) on a finite alphabet $A$, the joint mass function factorises:
\begin{align*}
p(x_1, \ldots, x_n) = \mathbb{P}(X_1 = x_1, \ldots, X_n = x_n) = \prod_{i=1}^{n} p(x_i),
\end{align*}
where $p(x) = \mathbb{P}(X_1 = x)$. Taking $-\frac{1}{n} \log_2$:
\begin{align*}
-\frac{1}{n} \log_2 p(X_1, \ldots, X_n) = -\frac{1}{n} \sum_{i=1}^{n} \log_2 p(X_i) = \frac{1}{n} \sum_{i=1}^{n} Y_i,
\end{align*}
where $Y_i := -\log_2 p(X_i)$ for $i \ge 1$.
[/step]
[step:Verify that $(Y_i)$ is i.i.d. with finite expectation equal to $H(X_1)$]
Since the $X_i$ are i.i.d. and $Y_i = -\log_2 p(X_i)$ is a fixed measurable function of $X_i$, the $(Y_i)_{i \ge 1}$ are also i.i.d. The finite-alphabet assumption means $A$ is finite, so $Y_1$ takes only finitely many values; in particular $\mathbb{E}[|Y_1|] < \infty$. Using the convention $0 \cdot \log_2 0 = 0$ to handle symbols of zero probability:
\begin{align*}
\mathbb{E}[Y_1] = \mathbb{E}[-\log_2 p(X_1)] = -\sum_{x \in A} p(x) \log_2 p(x) = H(X_1),
\end{align*}
which is the [entropy of $X_1$](/theorems/???).
[guided]
Two points deserve scrutiny. First, why are the $Y_i$ i.i.d.? The $X_i$ are i.i.d. by the Bernoulli source hypothesis, and $Y_i$ is obtained from $X_i$ by the single measurable map $y: A \to \mathbb{R}$, $x \mapsto -\log_2 p(x)$ (with value $0$ at any symbol of zero probability). Applying a common measurable map to an i.i.d. sequence preserves the i.i.d. property, so $(Y_i)$ is i.i.d. with the same distribution as $Y_1$.
Second, why is $\mathbb{E}[Y_1] = H(X_1)$? Because $A$ is finite, we may expand the expectation as a finite sum weighted by $p$:
\begin{align*}
\mathbb{E}[-\log_2 p(X_1)] = \sum_{x \in A} p(x) \cdot (-\log_2 p(x)) = -\sum_{x \in A} p(x) \log_2 p(x).
\end{align*}
The right-hand side is exactly the Shannon entropy $H(X_1)$. Symbols with $p(x) = 0$ contribute nothing under the standard convention $0 \cdot \log_2 0 = 0$, which is justified by $\lim_{t \downarrow 0} t \log_2 t = 0$. The finiteness of $H(X_1)$ follows from the finiteness of $A$ and the continuity of $t \mapsto -t \log_2 t$ on $[0, 1]$.
[/guided]
[/step]
[step:Apply the weak law of large numbers to deduce convergence in probability]
Since $(Y_i)_{i \ge 1}$ are i.i.d. with finite mean $\mathbb{E}[Y_1] = H(X_1)$, the [Weak Law of Large Numbers](/theorems/???) applies and gives
\begin{align*}
\frac{1}{n} \sum_{i=1}^{n} Y_i \xrightarrow{\mathbb{P}} H(X_1) \quad \text{as } n \to \infty.
\end{align*}
Combining with the decomposition from the first step:
\begin{align*}
-\frac{1}{n} \log_2 p(X_1, \ldots, X_n) \xrightarrow{\mathbb{P}} H(X_1).
\end{align*}
[/step]
[step:Conclude the AEP with constant $H(X_1)$]
Convergence in probability means: for every $\varepsilon > 0$,
\begin{align*}
\mathbb{P}\!\left(\left| -\frac{1}{n} \log_2 p(X_1, \ldots, X_n) - H(X_1) \right| < \varepsilon \right) \to 1.
\end{align*}
Defining the typical sets
\begin{align*}
T_n := \left\{ (x_1, \ldots, x_n) \in A^n : 2^{-n(H(X_1) + \varepsilon)} \le p(x_1, \ldots, x_n) \le 2^{-n(H(X_1) - \varepsilon)} \right\},
\end{align*}
the event $\{|{-\tfrac{1}{n}\log_2 p(X_1,\ldots,X_n)} - H(X_1)| < \varepsilon\}$ coincides with $\{(X_1, \ldots, X_n) \in T_n\}$, so $\mathbb{P}(T_n) \to 1$ and the defining bounds of $T_n$ hold by construction. This is the [Asymptotic Equipartition Property](/theorems/???) with constant $H(X_1)$, completing the proof.
[/step]
