[proofplan]
We prove both directions for a finite matroid. If $M$ is represented over $\mathbb{F}_2$, then deletion and contraction preserve representability, while the rank-$2$ uniform matroid $U_{2,4}$ on four elements cannot be represented over $\mathbb{F}_2$ because a simple rank-$2$ binary matroid has at most three points. Conversely, assume that $M$ has no $U_{2,4}$ minor. We use two standard finite-matroid inputs: the binary circuit symmetric-difference criterion and the independent circuit-difference excluded-minor theorem of Tutte, which says that failure of the circuit symmetric-difference property produces a $U_{2,4}$ minor. These facts give the symmetric-difference property and hence a representation over $\mathbb{F}_2$.
[/proofplan]
[step:Show that a binary matroid cannot have a $U_{2,4}$ minor]
Let $E(M)$ denote the ground set of the matroid $M$, and let $U_{2,4}$ denote the rank-$2$ uniform matroid on a four-element ground set. Assume that $M$ is binary. Thus there are an integer $r \geq 0$ and a matrix
$A \in \mathbb{F}_2^{r \times E(M)}$ whose columns are indexed by $E(M)$ and whose vector matroid is $M$.
Deletion of an element corresponds to deleting the corresponding column of $A$. Contraction of a non-loop element corresponds, after elementary row operations over $\mathbb{F}_2$ and deletion of the contracted column, to taking the represented quotient matroid. Loops are contracted by deleting zero columns. Hence every minor of $M$ is also representable over $\mathbb{F}_2$.
It remains to note that $U_{2,4}$ is not binary. Suppose, toward a contradiction, that $U_{2,4}$ is represented by a matrix
$B \in \mathbb{F}_2^{2 \times 4}$ of rank $2$. Since $U_{2,4}$ has no loops, no column of $B$ is the zero vector. Since every two-element subset of $U_{2,4}$ is independent, no two columns of $B$ are equal, because over $\mathbb{F}_2$ equal nonzero columns are linearly dependent. But $\mathbb{F}_2^2$ has exactly three nonzero vectors, namely $(1,0)$, $(0,1)$, and $(1,1)$. Thus a simple rank-$2$ binary matroid has at most three elements, contradicting the four elements of $U_{2,4}$. Therefore no binary matroid has a minor isomorphic to $U_{2,4}$.
[/step]
[step:Use the circuit symmetric-difference criterion for binary matroids]
We record the criterion used in the converse direction.
[claim:Binary circuit symmetric-difference criterion]
Here $E(N)$ denotes the ground set of the matroid $N$. A finite matroid $N$ is binary if and only if, for every pair of circuits $C_1, C_2 \subseteq E(N)$, the symmetric difference
\begin{align*}
C_1 \triangle C_2 := (C_1 \setminus C_2) \cup (C_2 \setminus C_1)
\end{align*}
is a disjoint union of circuits of $N$.
[/claim]
[proof]
First suppose that $N$ is represented over $\mathbb{F}_2$ by a matrix
$A \in \mathbb{F}_2^{r \times E(N)}$. For a subset $S \subseteq E(N)$, define its incidence vector
\begin{align*}
\mathbb{1}_S: E(N) \to \mathbb{F}_2
\end{align*}
by $\mathbb{1}_S(e)=1$ if $e \in S$ and $\mathbb{1}_S(e)=0$ otherwise. Since $C_1$ and $C_2$ are circuits, each is a minimally dependent set of columns. Over $\mathbb{F}_2$, every nonzero scalar is equal to $1$, so the dependence relation on a circuit has all coefficients equal to $1$. Hence the sums of their corresponding columns vanish:
\begin{align*}
\sum_{e \in C_1} A_e = 0
\end{align*}
and
\begin{align*}
\sum_{e \in C_2} A_e = 0.
\end{align*}
Adding these two equalities in characteristic $2$ gives
\begin{align*}
\sum_{e \in C_1 \triangle C_2} A_e = 0.
\end{align*}
Thus $C_1 \triangle C_2$ has zero column-sum. If it is nonempty, then it is dependent, so it contains a circuit $D_1$. Since $D_1$ is a circuit in a binary representation, the sum of the columns indexed by $D_1$ is zero. Therefore $(C_1 \triangle C_2) \setminus D_1=(C_1 \triangle C_2)\triangle D_1$ also has zero column-sum. If this remaining set is nonempty, it is again dependent and contains another circuit. Repeating this extraction and using finiteness of $E(N)$, we decompose $C_1 \triangle C_2$ as a disjoint union of circuits.
Conversely, assume that the symmetric difference of any two circuits of $N$ is a disjoint union of circuits. Choose a basis $B \subseteq E(N)$. For each element $e \in E(N) \setminus B$, let $C_B(e)$ denote the fundamental circuit of $e$ with respect to $B$, the unique circuit contained in $B \cup \{e\}$. Define a matrix
$A \in \mathbb{F}_2^{B \times E(N)}$
as follows: for $b \in B$, the $b$-column is the standard basis vector indexed by $b$; for $e \in E(N) \setminus B$, the $e$-column is the vector whose support is $C_B(e) \setminus \{e\}$.
Let $N_A$ be the vector matroid represented by $A$ over $\mathbb{F}_2$. By construction, $B$ is a basis of $N_A$, and for each $e \in E(N) \setminus B$, the fundamental circuit of $e$ with respect to $B$ in $N_A$ is exactly $C_B(e)$.
We use the following fundamental-circuit expansion lemma, and prove it here. If a finite matroid satisfies the symmetric-difference property, $B$ is a basis, and $C$ is a circuit, then
\begin{align*}
C = \triangle_{e \in C \setminus B} C_B(e),
\end{align*}
where the right-hand side is the iterated symmetric difference over the finite set $C \setminus B$.
Indeed, define
\begin{align*}
S := \triangle_{e \in C \setminus B} C_B(e).
\end{align*}
For each $e \in C \setminus B$, the fundamental circuit $C_B(e)$ is contained in $B \cup \{e\}$ and contains $e$, so its only element outside $B$ is $e$. Hence the non-basis part of $S$ is exactly $C \setminus B$. We need the two-circuit hypothesis in the slightly extended form that the symmetric difference of a finite disjoint union of circuits with one additional circuit is again a disjoint union of circuits. This follows by induction on the number of circuit components in the disjoint union: the empty union gives the additional circuit itself, one component is exactly the hypothesis, and the induction step first applies the induction hypothesis to all but one component and then applies the one-component case successively to the resulting circuit components. Applying this extension repeatedly to the finite family $(C_B(e))_{e \in C \setminus B}$ shows that $S$ is a disjoint union of circuits. Applying the same extension to the circuit components of $S$ together with the circuit $C$ shows that $S \triangle C$ is also a disjoint union of circuits. But $S$ and $C$ have the same elements outside $B$, so $S \triangle C \subseteq B$. Since $B$ is independent, it contains no circuit, and the only disjoint union of circuits contained in $B$ is the empty union. Thus $S \triangle C=\varnothing$, which proves $S=C$.
Now let $C$ be a circuit of $N$. Since $B$ is independent in $N$, the set $X := C \setminus B$ is nonempty. By the fundamental-circuit lemma,
\begin{align*}
C = \triangle_{e \in X} C_B(e).
\end{align*}
In the matrix $A$, the sum of the columns indexed by $C$ is zero: each non-basis column indexed by $e \in X$ contributes the vector with support $C_B(e) \setminus \{e\}$, and each basis column indexed by $b \in B$ contributes the standard basis vector indexed by $b$. The parity of the total support is exactly the parity of the symmetric difference $\triangle_{e \in X} C_B(e)=C$, so every coordinate cancels in $\mathbb{F}_2$. Therefore $C$ is dependent in $N_A$.
Conversely, let $C'$ be a circuit of $N_A$. The matroid $N_A$ is binary by construction, so it satisfies the already proved forward half of the symmetric-difference property. The basis $B$ and the fundamental circuits $C_B(e)$ are the same in $N$ and $N_A$ by construction. Applying the same fundamental-circuit lemma inside $N_A$ gives
\begin{align*}
C' = \triangle_{e \in C' \setminus B} C_B(e).
\end{align*}
The right-hand side is a disjoint union of circuits of $N$ by the assumed symmetric-difference property, so $C'$ is dependent in $N$. Thus every circuit of $N_A$ is dependent in $N$, and every circuit of $N$ is dependent in $N_A$. Hence the two matroids have the same dependent sets and therefore the same circuits. A finite matroid is determined by its circuits, so $N=N_A$, and $N$ is binary.
[/proof]
[guided]
The criterion formalises the idea that circuits in a binary representation add like vectors over $\mathbb{F}_2$. Suppose first that $N$ is represented over $\mathbb{F}_2$ by a matrix $A \in \mathbb{F}_2^{r \times E(N)}$, where $E(N)$ is the ground set and the columns are indexed by elements of $E(N)$. For a subset $S \subseteq E(N)$, define the incidence vector $\mathbb{1}_S:E(N)\to \mathbb{F}_2$ by setting $\mathbb{1}_S(e)=1$ for $e\in S$ and $\mathbb{1}_S(e)=0$ for $e\notin S$. If $C_1$ and $C_2$ are circuits, then each is a minimally dependent set of columns. In a representation over $\mathbb{F}_2$, the only nonzero scalar is $1$, so the unique dependence relation on each circuit has all coefficients equal to $1$. Thus the corresponding column-sums are zero:
\begin{align*}
\sum_{e\in C_1} A_e=0
\end{align*}
and
\begin{align*}
\sum_{e\in C_2} A_e=0.
\end{align*}
Adding these equalities in characteristic $2$ cancels the columns indexed by $C_1\cap C_2$ and leaves
\begin{align*}
\sum_{e\in C_1\triangle C_2} A_e=0.
\end{align*}
Thus $C_1\triangle C_2$ is dependent unless it is empty. If it is nonempty, choose a circuit $D_1\subseteq C_1\triangle C_2$. Since $D_1$ also has zero column-sum, the remaining set $(C_1\triangle C_2)\setminus D_1$ still has zero column-sum. Repeating this extraction terminates because $E(N)$ is finite, and the result is a disjoint union of circuits.
Conversely, assume that the symmetric difference of any two circuits of $N$ is a disjoint union of circuits. Choose a basis $B\subseteq E(N)$. For each $e\in E(N)\setminus B$, let $C_B(e)$ be the fundamental circuit of $e$ with respect to $B$, meaning the unique circuit contained in $B\cup\{e\}$. Define a matrix $A\in\mathbb{F}_2^{B\times E(N)}$ as follows: if $b\in B$, the $b$-column is the standard basis vector indexed by $b$; if $e\in E(N)\setminus B$, the $e$-column is the vector whose support is $C_B(e)\setminus\{e\}$. Let $N_A$ be the vector matroid represented by this matrix.
By construction, $B$ is a basis of $N_A$, and the fundamental circuit of each $e\in E(N)\setminus B$ relative to $B$ in $N_A$ is exactly $C_B(e)$. Now take a circuit $C$ of $N$. Since $B$ is independent, $X:=C\setminus B$ is nonempty. The fundamental-circuit expansion lemma says that, under the symmetric-difference hypothesis,
\begin{align*}
C=\triangle_{e\in X} C_B(e).
\end{align*}
Here is the proof of the lemma in the present notation. Define
\begin{align*}
S:=\triangle_{e\in X} C_B(e).
\end{align*}
Each fundamental circuit $C_B(e)$ is contained in $B\cup\{e\}$ and contains $e$, so $e$ is the only non-basis element of $C_B(e)$. Therefore the non-basis elements of $S$ are exactly the elements of $X=C\setminus B$. The hypothesis is stated for two circuits, so we first explain why it can be used for the finite symmetric differences appearing here. If $T$ is a finite disjoint union of circuits and $D$ is one more circuit, then $T\triangle D$ is again a disjoint union of circuits. We prove this by induction on the number of circuit components of $T$: the empty case gives $D$, the one-component case is exactly the assumed two-circuit property, and the induction step applies the induction hypothesis to all but one component of $T$ and then applies the one-component case successively to the circuit components obtained. Therefore repeated symmetric differences of finitely many circuits decompose into disjoint unions of circuits.
Applying this finite-extension argument to the family $(C_B(e))_{e\in X}$ proves that $S$ is a disjoint union of circuits. Applying it again to the circuit components of $S$ together with the circuit $C$ shows that $S\triangle C$ is also a disjoint union of circuits. Since $S$ and $C$ have the same non-basis elements, $S\triangle C\subseteq B$. But $B$ is independent, so no nonempty subset of $B$ contains a circuit. Hence $S\triangle C=\varnothing$, and therefore $S=C$. This is why the lemma is the right tool: it reconstructs an arbitrary circuit from the fundamental circuits that record the dependences created by adding the non-basis elements of that circuit to the basis.
Using the displayed identity, the sum of the columns of $A$ indexed by $C$ is zero. The non-basis columns indexed by $X$ contribute exactly the parity vector of $\triangle_{e\in X}(C_B(e)\setminus\{e\})$, while the basis columns indexed by $C\cap B$ contribute the matching standard basis vectors. Since $C=\triangle_{e\in X}C_B(e)$, every row coordinate occurs with even parity. Hence $C$ is dependent in $N_A$.
Now let $C'$ be a circuit of $N_A$. Because $N_A$ is binary by construction, the forward half already proved gives the symmetric-difference property in $N_A$. The matroids $N$ and $N_A$ have the same basis $B$ and the same fundamental circuits relative to $B$. Applying the same fundamental-circuit lemma in $N_A$ gives
\begin{align*}
C'=\triangle_{e\in C'\setminus B} C_B(e).
\end{align*}
The right-hand side is a disjoint union of circuits of $N$ by the hypothesis on $N$, so $C'$ is dependent in $N$. Thus every circuit of $N$ is dependent in $N_A$, and every circuit of $N_A$ is dependent in $N$. If a set is dependent in $N$, then it contains a circuit of $N$, hence contains a dependent set of $N_A$ and is dependent in $N_A$. The same argument with $N$ and $N_A$ interchanged shows that every dependent set of $N_A$ is dependent in $N$. Therefore the two matroids have the same dependent sets and hence the same circuits, so the represented matroid $N_A$ equals $N$. This proves that $N$ is binary.
[/guided]
[/step]
[step:Derive the symmetric-difference property from Tutte's circuit-difference minor lemma]
Assume that $M$ has no minor isomorphic to $U_{2,4}$. We invoke Tutte's circuit-difference minor lemma, a finite-matroid result whose statement is independent of representability: if a finite matroid $N$ has circuits $C_1,C_2\subseteq E(N)$ such that $C_1\triangle C_2$ is not a disjoint union of circuits of $N$, then $N$ has a minor isomorphic to $U_{2,4}$. This lemma is not the binary excluded-minor characterization itself; it is a purely circuit-minor statement, and the only use of binary representability in this proof is the circuit symmetric-difference criterion proved in the previous step.
The hypotheses of Tutte's circuit-difference minor lemma are satisfied with $N=M$ if the circuit symmetric-difference property fails for $M$: the matroid $M$ is finite, and the failure supplies circuits $C_1,C_2\subseteq E(M)$ for which $C_1\triangle C_2$ is not a disjoint union of circuits. The lemma would then give a minor of $M$ isomorphic to $U_{2,4}$, contradicting the standing hypothesis. Therefore no such pair of circuits exists. Hence for every pair of circuits $C_1,C_2$ of $M$, the symmetric difference $C_1\triangle C_2$ is a disjoint union of circuits.
[/step]
[step:Construct the binary representation and conclude]
By the previous step, $M$ satisfies the circuit symmetric-difference property. Applying the binary circuit symmetric-difference criterion with $N=M$, we obtain a matrix over $\mathbb{F}_2$ whose vector matroid is $M$. Hence $M$ is binary.
Combining this converse implication with the forward implication proved above, $M$ is binary if and only if $M$ has no minor isomorphic to $U_{2,4}$.
[/step]