[guided]We want to understand exactly which tangent vectors are killed by the restricted two-form $\alpha=i_\mu^*\omega$. This is the key point because quotienting by $G_\mu$ removes precisely the tangent directions to the $G_\mu$-orbits. Thus the desired form on the quotient can be nondegenerate only if the degeneracy of $\alpha$ is exactly vertical.
Fix $x\in N=J^{-1}(\mu)$. Since $\mu$ is a regular value of $J:M\to\mathfrak g^*$, the tangent space to the level set is
\begin{align*}
T_xN=\ker dJ_x.
\end{align*}
For each $\xi\in\mathfrak g$, the notation $\xi_M\in\mathfrak X(M)$ denotes the infinitesimal action vector field defined by $\xi_M(z)=\left.\frac{d}{dt}\right|_{t=0}\exp(t\xi)\cdot z\in T_zM$ for $z\in M$. Let $\xi\in\mathfrak g_\mu$, and let $v\in T_xN$. Before evaluating $\alpha_x$ on $\xi_M(x)$, we verify that this infinitesimal action vector is tangent to $N$. Define the curve $c:\mathbb R\to M$ by $c(t)=\exp(t\xi)\cdot x$. Equivariance gives $J(c(t))=\operatorname{Ad}_{\exp(t\xi)}^*\mu$, so
\begin{align*}
\left.\frac{d}{dt}\right|_{t=0}J(c(t))=\operatorname{ad}_\xi^*\mu=0
\end{align*}
because $\xi\in\mathfrak g_\mu$. Hence $\xi_M(x)=c'(0)\in\ker dJ_x=T_xN$. The function associated to $\xi$ by the momentum map is $J_\xi:M\to\mathbb R$, defined by $J_\xi(y)=J(y)(\xi)$. We are using the convention that equivariance means $J(g\cdot y)=\operatorname{Ad}_g^*J(y)$ and the Hamiltonian momentum-map identity is
\begin{align*}
\iota_{\xi_M}\omega=dJ_\xi.
\end{align*}
Evaluating this identity at $x$ on the vector $v$ gives
\begin{align*}
\alpha_x(\xi_M(x),v)=\omega_x(\xi_M(x),v)=d(J_\xi)_x(v)=dJ_x(v)(\xi).
\end{align*}
Since $v\in T_xN=\ker dJ_x$, the element $dJ_x(v)\in\mathfrak g^*$ is zero, so $dJ_x(v)(\xi)=0$. Therefore every vector tangent to a $G_\mu$-orbit lies in $\ker\alpha_x$.
The harder direction is to show that there are no additional kernel vectors. Suppose $v\in\ker\alpha_x$. Then $v\in T_xN$ and
\begin{align*}
\omega_x(v,w)=0
\end{align*}
for every $w\in T_xN=\ker dJ_x$. Define the infinitesimal action map $L_x:\mathfrak g\to T_xM$ by $L_x(\xi)=\xi_M(x)$, and define
\begin{align*}
\operatorname{Range}L_x:=\{L_x(\xi):\xi\in\mathfrak g\}\subset T_xM.
\end{align*}
For every $w\in T_xM$ and every $\xi\in\mathfrak g$, the momentum map identity gives
\begin{align*}
dJ_x(w)(\xi)=d(J_\xi)_x(w)=\omega_x(L_x(\xi),w).
\end{align*}
This formula says exactly that a vector $w$ lies in $\ker dJ_x$ if and only if it is symplectically orthogonal to every infinitesimal orbit vector $L_x(\xi)$. Hence
\begin{align*}
\ker dJ_x=(\operatorname{Range}L_x)^\omega.
\end{align*}
Now use finite-dimensional symplectic linear algebra. Since $dJ_x$ is surjective, $\ker dJ_x$ has codimension $\dim\mathfrak g$. We also need to justify that $\operatorname{Range}L_x$ has dimension $\dim\mathfrak g$, which is equivalent to injectivity of $L_x$. Suppose $L_x(\xi)=0$. Then for every $w\in T_xM$,
\begin{align*}
dJ_x(w)(\xi)=\omega_x(L_x(\xi),w)=0.
\end{align*}
Because $dJ_x:T_xM\to\mathfrak g^*$ is surjective, every $\lambda\in\mathfrak g^*$ has the form $\lambda=dJ_x(w)$ for some $w\in T_xM$. Hence $\lambda(\xi)=0$ for every $\lambda\in\mathfrak g^*$, so $\xi=0$. Thus $L_x$ is injective and $\dim\operatorname{Range}L_x=\dim\mathfrak g$. Therefore taking the symplectic orthogonal twice gives
\begin{align*}
(\ker dJ_x)^\omega=((\operatorname{Range}L_x)^\omega)^\omega=\operatorname{Range}L_x.
\end{align*}
Because $v$ annihilates $\ker dJ_x$ under $\omega_x$, we have $v\in(\ker dJ_x)^\omega$, so there exists $\xi\in\mathfrak g$ such that
\begin{align*}
v=\xi_M(x).
\end{align*}
It remains to prove that this $\xi$ actually belongs to $\mathfrak g_\mu$, not just to $\mathfrak g$. Since $v\in T_xN=\ker dJ_x$, we have $dJ_x(v)=0$, and substituting $v=\xi_M(x)$ gives $dJ_x(\xi_M(x))=0$. To compute this derivative, define the curve $c:\mathbb R\to M$ by $c(t)=\exp(t\xi)\cdot x$. Then $c(0)=x$ and $c'(0)=\xi_M(x)$. Equivariance of $J$ gives
\begin{align*}
J(c(t))=\operatorname{Ad}_{\exp(t\xi)}^*J(x)
\end{align*}
for all $t$ for which the expression is defined. Differentiating at $t=0$ and using $J(x)=\mu$ gives
\begin{align*}
0=dJ_x(\xi_M(x))=\left.\frac{d}{dt}\right|_{t=0}\operatorname{Ad}_{\exp(t\xi)}^*J(x)=\operatorname{ad}_\xi^*\mu.
\end{align*}
Thus $\xi\in\mathfrak g_\mu$. Therefore
\begin{align*}
\ker\alpha_x=\{\xi_M(x):\xi\in\mathfrak g_\mu\},
\end{align*}
which is exactly the vertical tangent space of the quotient projection.[/guided]