Two coins can both be fair and still tell completely different stories. In one model the second toss is a fresh experiment; in another model it is a copy of the first toss; in a third model it is forced to be the opposite result. The marginal probability of heads is $1/2$ in all three models, so marginal probabilities do not tell us whether observations carry new information.

Independence is the condition that rules out hidden information flow. It is the reason products of probabilities appear in repeated trials, the reason expectations factor, and the reason long averages stabilise. Without it, the same one-dimensional distributions can produce very different joint behaviour.

[example: Two Fair Coins with Different Dependence] Let $(\Omega,\mathcal F,\mathbb P)$ be the uniform probability space on $\{HH,HT,TH,TT\}$. Let $X$ be the first coordinate and $Y$ the second coordinate. Since each of the four outcomes has probability $1/4$, \begin{align*} \{X=H\}&=\{HH,HT\},& \{Y=H\}&=\{HH,TH\},& \{X=H,Y=H\}&=\{HH\}. \end{align*} Therefore \begin{align*} \mathbb P(X=H)&=\mathbb P(\{HH,HT\}) =\mathbb P(\{HH\})+\mathbb P(\{HT\}) =\frac14+\frac14 =\frac12,\\ \mathbb P(Y=H)&=\mathbb P(\{HH,TH\}) =\mathbb P(\{HH\})+\mathbb P(\{TH\}) =\frac14+\frac14 =\frac12,\\ \mathbb P(X=H,Y=H)&=\mathbb P(\{HH\}) =\frac14. \end{align*} Thus \begin{align*} \mathbb P(X=H)\mathbb P(Y=H) =\frac12\cdot\frac12 =\frac14 =\mathbb P(X=H,Y=H), \end{align*} so the heads event for the first coordinate factors from the heads event for the second coordinate in this product model. Now let $(\Omega',\mathcal F',\mathbb P')$ be uniform on $\{H,T\}$, and define $X'(\omega)=\omega$ and $Y'(\omega)=\omega$. Then \begin{align*} \{X'=H\}&=\{H\},& \{Y'=H\}&=\{H\},& \{X'=H,Y'=H\}&=\{H\}\cap\{H\}=\{H\}. \end{align*} Since $\mathbb P'(\{H\})=1/2$, \begin{align*} \mathbb P'(X'=H)&=\frac12,& \mathbb P'(Y'=H)&=\frac12,& \mathbb P'(X'=H,Y'=H)&=\frac12. \end{align*} But \begin{align*} \mathbb P'(X'=H)\mathbb P'(Y'=H) =\frac12\cdot\frac12 =\frac14 \ne \frac12 =\mathbb P'(X'=H,Y'=H). \end{align*} Both $X'$ and $Y'$ are fair, but observing $X'=H$ forces $Y'=H$; the first model describes two fresh tosses, while the second describes the same toss recorded twice. [/example]

This example gives the central warning for the whole chapter: independence is a property of a joint model, not of separate distributions. We begin with events, move to random variables and sigma-algebras, then reformulate independence through product measures and expectation factorisation.

## Definition

The most concrete question is whether two events influence each other. If $B$ has positive probability, the phrase "knowing $B$ does not change the chance of $A$" means $\mathbb P(A\mid B)=\mathbb P(A)$. Multiplying by $\mathbb P(B)$ gives a formula that remains meaningful when $\mathbb P(B)=0$.

[definition: Independent Events] Let $(\Omega,\mathcal F,\mathbb P)$ be a probability space. Events $A,B\in\mathcal F$ are independent if \begin{align*} \mathbb P(A\cap B)=\mathbb P(A)\mathbb P(B). \end{align*} [/definition]

Independence should not be confused with disjointness. Disjoint events cannot occur together, so learning one of them occurred usually gives decisive information about the other. The following computation shows the failure directly.

[example: Disjoint Events Are Usually Dependent] Let $\Omega=\{1,2,3,4,5,6\}$ with the uniform probability measure, so each singleton has probability $1/6$. Let \begin{align*} A=\{1,2,3\}, \qquad B=\{4,5,6\}. \end{align*} The two sets have no common outcome, hence \begin{align*} A\cap B &=\{1,2,3\}\cap\{4,5,6\} =\varnothing, \end{align*} and therefore \begin{align*} \mathbb P(A\cap B) =\mathbb P(\varnothing) =0. \end{align*} On the other hand, \begin{align*} \mathbb P(A) &=\mathbb P(\{1,2,3\}) =\mathbb P(\{1\})+\mathbb P(\{2\})+\mathbb P(\{3\}) \\ &=\frac16+\frac16+\frac16 =\frac36 =\frac12, \end{align*} and similarly \begin{align*} \mathbb P(B) &=\mathbb P(\{4,5,6\}) =\mathbb P(\{4\})+\mathbb P(\{5\})+\mathbb P(\{6\}) \\ &=\frac16+\frac16+\frac16 =\frac36 =\frac12. \end{align*} Thus \begin{align*} \mathbb P(A)\mathbb P(B) =\frac12\cdot\frac12 =\frac14 \ne 0 =\mathbb P(A\cap B). \end{align*} So $A$ and $B$ are not independent: observing that $B$ occurred rules out every outcome in $A$. [/example]

## Families and Random Variables

### Event Families

Many probabilistic models involve more than two events, and the problem is not solved by checking pairs. A hidden relation can appear only when several events are considered together. The next definition imposes the finite intersection factorisation needed for a whole family to behave like separate sources of randomness.

[definition: Mutual Independence of Events] Let $(\Omega,\mathcal F,\mathbb P)$ be a probability space, and let $(A_i)_{i\in I}$ be a family of events in $\mathcal F$. The family $(A_i)_{i\in I}$ is mutually independent if, for every finite subset $J\subset I$, \begin{align*} \mathbb P\left(\bigcap_{j\in J}A_j\right)=\prod_{j\in J}\mathbb P(A_j). \end{align*} [/definition]

The finite-subset condition is the real content of mutual independence. It prevents a hidden relation from appearing only when several events are considered together. The next example is the standard small model where pairwise independence survives but mutual independence fails.

[example: Pairwise Independence Without Mutual Independence] Let $\Omega=\{00,01,10,11\}$ with the uniform probability measure, so each singleton has probability $1/4$. Define \begin{align*} A&=\{00,01\},& B&=\{00,10\},& C&=\{00,11\}. \end{align*} Then \begin{align*} \mathbb P(A) &=\mathbb P(\{00,01\}) =\mathbb P(\{00\})+\mathbb P(\{01\}) =\frac14+\frac14 =\frac12,\\ \mathbb P(B) &=\mathbb P(\{00,10\}) =\mathbb P(\{00\})+\mathbb P(\{10\}) =\frac14+\frac14 =\frac12,\\ \mathbb P(C) &=\mathbb P(\{00,11\}) =\mathbb P(\{00\})+\mathbb P(\{11\}) =\frac14+\frac14 =\frac12. \end{align*} The pairwise intersections are \begin{align*} A\cap B &=\{00,01\}\cap\{00,10\} =\{00\},\\ A\cap C &=\{00,01\}\cap\{00,11\} =\{00\},\\ B\cap C &=\{00,10\}\cap\{00,11\} =\{00\}. \end{align*} Therefore \begin{align*} \mathbb P(A\cap B) &=\mathbb P(\{00\}) =\frac14 =\frac12\cdot\frac12 =\mathbb P(A)\mathbb P(B),\\ \mathbb P(A\cap C) &=\mathbb P(\{00\}) =\frac14 =\frac12\cdot\frac12 =\mathbb P(A)\mathbb P(C),\\ \mathbb P(B\cap C) &=\mathbb P(\{00\}) =\frac14 =\frac12\cdot\frac12 =\mathbb P(B)\mathbb P(C). \end{align*} Thus every pair among $A,B,C$ is independent. However, the triple intersection is \begin{align*} A\cap B\cap C &=\{00,01\}\cap\{00,10\}\cap\{00,11\} =\{00\}, \end{align*} so \begin{align*} \mathbb P(A\cap B\cap C) &=\mathbb P(\{00\}) =\frac14. \end{align*} On the other hand, \begin{align*} \mathbb P(A)\mathbb P(B)\mathbb P(C) =\frac12\cdot\frac12\cdot\frac12 =\frac18. \end{align*} Since \begin{align*} \mathbb P(A\cap B\cap C) =\frac14 \ne \frac18 =\mathbb P(A)\mathbb P(B)\mathbb P(C), \end{align*} the three events are pairwise independent but not mutually independent. [/example]

### Random Variables

Events are yes-or-no observations, but random variables carry many observations at once. To ask whether random variables are independent, we must ask whether all events determined by one variable are independent of all events determined by the other. This motivates the sigma-algebra generated by a random variable.

[definition: Sigma-Algebra Generated by a Random Variable] Let $(\Omega,\mathcal F,\mathbb P)$ be a probability space, let $(E,\mathcal E)$ be a measurable space, and let $X:(\Omega,\mathcal F)\to(E,\mathcal E)$ be a random variable. The sigma-algebra generated by $X$ is \begin{align*} \sigma(X)=\{X^{-1}(A):A\in\mathcal E\}. \end{align*} [/definition]

The generated sigma-algebra contains exactly the events whose truth can be decided after observing $X$. The next problem is to turn that information content into an independence condition for variables rather than individual events. The definition below requires every event generated by one variable to factor from every compatible event generated by the others.

[definition: Independent Random Variables] Let $(\Omega,\mathcal F,\mathbb P)$ be a probability space. Let $X_i:(\Omega,\mathcal F)\to(E_i,\mathcal E_i)$ be random variables indexed by $i\in I$. The family $(X_i)_{i\in I}$ is independent if the family of sigma-algebras $(\sigma(X_i))_{i\in I}$ is mutually independent. [/definition]