Counting random events by listing all their probabilities is honest but quickly becomes unwieldy. If $X$ counts the number of successes in a collection of trials, then the distribution of $X$ is a sequence $p_X(0),p_X(1),p_X(2),\dots$. When another independent count $Y$ is added, the probability of $X+Y=n$ is a convolution of two such sequences, and repeated convolution is exactly the kind of bookkeeping that hides the structure of the problem.

The probability [generating function](/page/Generating%20Function) replaces the sequence by a [power series](/page/Power%20Series). The coefficient of $z^n$ remembers $\mathbb P(X=n)$, while multiplication of power series remembers convolution. This is the same idea that makes ordinary generating functions useful in combinatorics, but the coefficients are probabilities, so the series also carries normalization, moments, and distributional information.

[example: Two Bernoulli Trials Before the Definition] Let $X_1,X_2$ be independent random variables with $X_i\sim\operatorname{Ber}(p)$, so $\mathbb P(X_i=1)=p$ and $\mathbb P(X_i=0)=1-p$. Since each $X_i$ takes only the values $0$ and $1$, the sum $X_1+X_2$ can take only the values $0,1,2$. For the value $0$, both trials must be $0$, and independence gives \begin{align*} \mathbb P(X_1+X_2=0)=\mathbb P(X_1=0,X_2=0)=\mathbb P(X_1=0)\mathbb P(X_2=0)=(1-p)^2. \end{align*} For the value $1$, exactly one trial must be $1$, so the two disjoint cases are $(X_1,X_2)=(1,0)$ and $(X_1,X_2)=(0,1)$. Hence \begin{align*} \mathbb P(X_1+X_2=1)=\mathbb P(X_1=1,X_2=0)+\mathbb P(X_1=0,X_2=1). \end{align*} Using independence in each term, \begin{align*} \mathbb P(X_1+X_2=1)=p(1-p)+(1-p)p=2p(1-p). \end{align*} For the value $2$, both trials must be $1$, so \begin{align*} \mathbb P(X_1+X_2=2)=\mathbb P(X_1=1,X_2=1)=\mathbb P(X_1=1)\mathbb P(X_2=1)=p^2. \end{align*} The polynomial $(1-p)+pz$ stores the two probabilities of one Bernoulli trial as the constant coefficient and the coefficient of $z$. Multiplying the two copies gives \begin{align*} ((1-p)+pz)^2=((1-p)+pz)((1-p)+pz). \end{align*} Expanding term by term, \begin{align*} ((1-p)+pz)((1-p)+pz)=(1-p)^2+(1-p)pz+pz(1-p)+p^2z^2. \end{align*} Since $(1-p)pz+pz(1-p)=2p(1-p)z$, this becomes \begin{align*} ((1-p)+pz)^2=(1-p)^2+2p(1-p)z+p^2z^2. \end{align*} The coefficient of $z^0$ is $\mathbb P(X_1+X_2=0)$, the coefficient of $z^1$ is $\mathbb P(X_1+X_2=1)$, and the coefficient of $z^2$ is $\mathbb P(X_1+X_2=2)$. The point is not the algebra for two trials, but that multiplication is already doing the same bookkeeping as the distribution of an independent sum. [/example]

This example suggests that the variable $z$ is a placeholder for the count, not a [random variable](/page/Random%20Variable). The power series is a compact encoding of a distribution on $\{0,1,2,\dots\}$, and the probabilistic content is recovered from coefficients and derivatives.

## Definition

The construction begins with random variables whose values are non-negative integers. This restriction is not cosmetic: powers $z^k$ naturally index counts $k=0,1,2,\dots$, and the resulting series has no need for negative powers or fractional exponents.

[definition: Probability Generating Function] Let $(\Omega,\mathcal F,\mathbb P)$ be a [probability space](/page/Probability%20Space), and let $X: \Omega \to \{0,1,2,\dots\}$ be a random variable. Write \begin{align*} p_X(k)=\mathbb P(X=k), \qquad k\in \{0,1,2,\dots\}. \end{align*} The probability generating function of $X$ is the function $G_X:\{z\in\mathbb C:|z|\le1\}\to\mathbb C$ defined by \begin{align*} G_X(z)=\mathbb E[z^X]=\sum_{k=0}^{\infty}p_X(k)z^k. \end{align*} [/definition]

For $|z|\le1$, the series is absolutely bounded by $\sum_{k=0}^{\infty}p_X(k)=1$, so the definition always makes sense on the closed unit disk. On the real interval $[0,1]$, the pgf takes values in $[0,1]$ and has the direct probabilistic interpretation used in many arguments. Sometimes the same formula defines an [analytic function](/page/Analytic%20Function) on a larger disk, and that analytic viewpoint gives sharper coefficient and limit tools.

Because the coefficients are probabilities, the value at $1$ is fixed:

\begin{align*} G_X(1)=\sum_{k=0}^{\infty}p_X(k)=1. \end{align*}

Also $G_X(0)=p_X(0)$, so the first visible value already records the atom at zero.

A pgf is useful only if the encoding can be inverted one probability at a time. Values such as $G_X(0)$ recover only special atoms, and derivatives recover moments rather than the full mass function. To recover the law itself, we need a notation for isolating the coefficient of a specified power of $z$.

[definition: Coefficient Extraction] Let $\mathcal A$ be the class of formal power series in $z$ with complex coefficients, together with power series in $z$ that converge in a neighbourhood of $0$. For each $n\in\{0,1,2,\dots\}$, coefficient extraction is the map $[z^n]:\mathcal A\to\mathbb C$ defined by \begin{align*} [z^n]\sum_{k=0}^{\infty}a_kz^k=a_n. \end{align*} [/definition]

With this notation, the probabilities encoded by a probability generating function are $\mathbb P(X=n)=[z^n]G_X(z)$. This is the first reason pgfs are faithful encodings rather than summaries: no probability is discarded.

The coefficient operation is algebraic, but many pgf arguments also use limits and derivatives. To know where those analytic manipulations are valid, we need the convergence disk of the underlying power series.

[definition: Radius of Convergence] Let $\mathcal P$ be the class of complex power series in $z$. The [radius of convergence](/theorems/262) is the map \begin{align*} R:\mathcal P\to[0,\infty], \end{align*} where $R\left(\sum_{k=0}^{\infty}a_kz^k\right)$ is the unique number such that the series converges absolutely for $|z|<R$ and diverges for $|z|>R$. [/definition]

For a probability generating function, $R\ge 1$ because $\sum_k p_X(k)|z|^k\le 1$ whenever $|z|\le 1$. If $R>1$, then for every $\rho$ with $1<\rho<R$ the weighted sum $\sum_{k=0}^{\infty}p_X(k)\rho^k$ is finite, so the probabilities have exponential control at every rate strictly below $R$. If $R=1$, the distribution may still have finite moments, but the [analytic continuation](/page/Analytic%20Continuation) is more delicate.

## Extracting Probabilities and Moments

The first task after defining a pgf is to recover what it stores. Coefficients recover the mass function; derivatives at $1$ recover factorial moments, which are often better adapted to counting than ordinary powers.

### Recovering the Law