[proofplan]
The proof is a direct matrix calculation. For an arbitrary vector $x \in \mathbb{R}^n$, expand the transpose of $Qx$ using the transpose rule for a matrix-vector product. Associativity of matrix multiplication then identifies the resulting scalar with $x^\top(Q^\top A Q)x$, and the definition of $B$ gives the desired expression.
[/proofplan]
[step:Expand the transformed quadratic form using the transpose product rule]
Fix $x \in \mathbb{R}^n$. Since $Q \in O(n)$, in particular $Q \in \mathbb{R}^{n \times n}$, so $Qx \in \mathbb{R}^n$. The transpose rule for matrix products gives
\begin{align*}
(Qx)^\top = x^\top Q^\top.
\end{align*}
Therefore, substituting this identity into the quadratic expression and using associativity of matrix multiplication,
\begin{align*}
(Qx)^\top A(Qx) = (x^\top Q^\top)A(Qx).
\end{align*}
Hence
\begin{align*}
(Qx)^\top A(Qx) = x^\top Q^\top A Qx.
\end{align*}
[guided]
Fix $x \in \mathbb{R}^n$. The first point is only a dimension check: $Q \in O(n)$ means that $Q$ is a real $n \times n$ [orthogonal matrix](/page/Orthogonal%20Matrix), so the product $Qx$ is a vector in $\mathbb{R}^n$, and the expression $(Qx)^\top A(Qx)$ is a well-defined scalar matrix product.
We now expand the transpose of $Qx$. The transpose rule for a product says that the transpose reverses the order of multiplication. Applied to the product of the matrix $Q \in \mathbb{R}^{n \times n}$ and the column vector $x \in \mathbb{R}^n$, it gives
\begin{align*}
(Qx)^\top = x^\top Q^\top.
\end{align*}
Substituting this into the left-hand side gives
\begin{align*}
(Qx)^\top A(Qx) = (x^\top Q^\top)A(Qx).
\end{align*}
Finally, matrix multiplication is associative whenever the dimensions match. Here the dimensions are compatible because $x^\top \in \mathbb{R}^{1 \times n}$, $Q^\top \in \mathbb{R}^{n \times n}$, $A \in \mathbb{R}^{n \times n}$, $Q \in \mathbb{R}^{n \times n}$, and $x \in \mathbb{R}^{n \times 1}$. Thus we may remove the intermediate parentheses and write
\begin{align*}
(Qx)^\top A(Qx) = x^\top Q^\top A Qx.
\end{align*}
[/guided]
[/step]
[step:Substitute the congruent matrix and conclude the identity]
By definition, $B = Q^\top A Q$. Substituting this into the expression obtained above gives
\begin{align*}
(Qx)^\top A(Qx) = x^\top Bx.
\end{align*}
Since $x \in \mathbb{R}^n$ was arbitrary, the identity holds for every $x \in \mathbb{R}^n$. This proves the theorem.
[/step]