[proofplan] We compute the mean and covariance of the transformed random vector $Z=QX$ directly from the definitions, using linearity of expectation and the identity $Q^\top Q=I_p$. Once the covariance transformation formula is obtained, we insert $Q^\top Q=I_p$ into the eigenvalue equation for $\Sigma$. Orthogonality also ensures that $Q\gamma_k$ is nonzero, so the transformed vector is a genuine eigenvector rather than the zero vector. [/proofplan] [step:Compute the mean and covariance after applying the orthogonal matrix] Define the map \begin{align*} Z:(\Omega,\mathcal F)&\to(\mathbb R^p,\mathcal B(\mathbb R^p))\\ \omega&\mapsto QX(\omega). \end{align*} Since $Q:\mathbb R^p\to\mathbb R^p$ is linear and hence continuous, $Z$ is a measurable random vector. Also, \begin{align*} |Z(\omega)|^2 = |QX(\omega)|^2 = X(\omega)^\top Q^\top QX(\omega)=|X(\omega)|^2 \end{align*} for every $\omega\in\Omega$, so $Z$ has finite second moment because $X$ has finite second moment. By linearity of expectation for integrable vector-valued random variables, \begin{align*} \mu_Z :=\mathbb E[Z] =\mathbb E[QX] =Q\mathbb E[X] =Q\mu_X. \end{align*} Therefore \begin{align*} Z-\mu_Z=QX-Q\mu_X=Q(X-\mu_X). \end{align*} Using the definition of covariance and the matrix identity $(AB)^\top=B^\top A^\top$, we obtain \begin{align*} \operatorname{Cov}(Z) &:=\mathbb E[(Z-\mu_Z)(Z-\mu_Z)^\top]\\ &=\mathbb E[Q(X-\mu_X)(X-\mu_X)^\top Q^\top]\\ &=Q\,\mathbb E[(X-\mu_X)(X-\mu_X)^\top]\,Q^\top\\ &=Q\Sigma Q^\top. \end{align*} [guided] We first verify that the transformed object is again a legitimate random vector with finite second moment. Define \begin{align*} Z:(\Omega,\mathcal F)&\to(\mathbb R^p,\mathcal B(\mathbb R^p))\\ \omega&\mapsto QX(\omega). \end{align*} The map $x\mapsto Qx$ from $\mathbb R^p$ to $\mathbb R^p$ is linear, hence continuous and Borel measurable. Since $X$ is measurable, the composition $Z=Q\circ X$ is measurable. The finite second moment condition is preserved because $Q$ is orthogonal. For every $\omega\in\Omega$, \begin{align*} |Z(\omega)|^2 =|QX(\omega)|^2 =(QX(\omega))^\top(QX(\omega)) =X(\omega)^\top Q^\top QX(\omega) =X(\omega)^\top X(\omega) =|X(\omega)|^2. \end{align*} Since $\mathbb E[|X|^2]<\infty$, this equality gives $\mathbb E[|Z|^2]<\infty$. Now compute the mean. Because $Q$ is a fixed matrix and $X$ is integrable, linearity of expectation gives \begin{align*} \mu_Z :=\mathbb E[Z] =\mathbb E[QX] =Q\mathbb E[X] =Q\mu_X. \end{align*} Thus the centered transformed vector is \begin{align*} Z-\mu_Z=QX-Q\mu_X=Q(X-\mu_X). \end{align*} Substituting this identity into the covariance definition gives \begin{align*} \operatorname{Cov}(Z) &:=\mathbb E[(Z-\mu_Z)(Z-\mu_Z)^\top]\\ &=\mathbb E[(Q(X-\mu_X))(Q(X-\mu_X))^\top]. \end{align*} Using $(AB)^\top=B^\top A^\top$, the matrix inside the expectation is \begin{align*} (Q(X-\mu_X))(Q(X-\mu_X))^\top =Q(X-\mu_X)(X-\mu_X)^\top Q^\top. \end{align*} The matrices $Q$ and $Q^\top$ are deterministic, so they factor outside the expectation: \begin{align*} \operatorname{Cov}(Z) &=Q\,\mathbb E[(X-\mu_X)(X-\mu_X)^\top]\,Q^\top\\ &=Q\Sigma Q^\top. \end{align*} This proves the covariance transformation formula. [/guided] [/step] [step:Transport each covariance eigenvector by the same orthogonal matrix] Let $\gamma_k\in\mathbb R^p\setminus\{0\}$ and $\lambda_k\in\mathbb R$ satisfy \begin{align*} \Sigma\gamma_k=\lambda_k\gamma_k. \end{align*} Since $Q^\top Q=I_p$, we have \begin{align*} (Q\Sigma Q^\top)(Q\gamma_k) =Q\Sigma(Q^\top Q)\gamma_k =Q\Sigma\gamma_k =Q(\lambda_k\gamma_k) =\lambda_k Q\gamma_k. \end{align*} It remains only to check that $Q\gamma_k$ is nonzero. Orthogonality gives \begin{align*} |Q\gamma_k|^2 =\gamma_k^\top Q^\top Q\gamma_k =\gamma_k^\top\gamma_k =|\gamma_k|^2>0. \end{align*} Thus $Q\gamma_k$ is an eigenvector of $Q\Sigma Q^\top$ with eigenvalue $\lambda_k$. This proves the claimed orthogonal equivariance of covariance principal components. [/step]