[step:Expand $F(x^* + h) - F(x^*)$ using bilinearity and symmetry of $A$]Let $h \in \mathbb{R}^n$ be arbitrary. Define $x^* := A^{-1}b$, so that $Ax^* = b$. Using the bilinearity of the inner product and the symmetry $A = A^\top$:
\begin{align*}
F(x^* + h) &= \frac{1}{2}\langle x^* + h, A(x^* + h) \rangle - \langle b, x^* + h \rangle \\
&= \frac{1}{2}\langle x^*, Ax^* \rangle + \frac{1}{2}\langle x^*, Ah \rangle + \frac{1}{2}\langle h, Ax^* \rangle + \frac{1}{2}\langle h, Ah \rangle - \langle b, x^* \rangle - \langle b, h \rangle.
\end{align*}
Since $A$ is symmetric, $\langle x^*, Ah \rangle = \langle Ax^*, h \rangle = \langle b, h \rangle$, so the two cross terms combine:
\begin{align*}
\frac{1}{2}\langle x^*, Ah \rangle + \frac{1}{2}\langle h, Ax^* \rangle = \frac{1}{2}\langle b, h \rangle + \frac{1}{2}\langle h, b \rangle = \langle b, h \rangle.
\end{align*}
Subtracting $F(x^*) = \frac{1}{2}\langle x^*, Ax^* \rangle - \langle b, x^* \rangle$:
\begin{align*}
F(x^* + h) - F(x^*) = \langle b, h \rangle + \frac{1}{2}\langle h, Ah \rangle - \langle b, h \rangle = \frac{1}{2}\langle h, Ah \rangle.
\end{align*}[/step]