[proofplan] We use the defining property of change-of-basis matrices: $P_{\mathcal C \leftarrow \mathcal B}$ converts $\mathcal B$-coordinate columns into $\mathcal C$-coordinate columns, while $P_{\mathcal B \leftarrow \mathcal C}$ converts in the reverse direction. Composing these two conversions sends every coordinate column back to itself. Since coordinate columns in an ordered basis range over all of $k^n$, both products of the two matrices are the identity matrix, which proves invertibility and identifies the inverse. [/proofplan]

[step:Declare the coordinate maps and the two change-of-basis matrices]For each ordered basis, define the coordinate maps \begin{align*} \operatorname{coord}_{\mathcal B}:V\to k^n,\quad v\mapsto [v]_{\mathcal B} \end{align*} and \begin{align*} \operatorname{coord}_{\mathcal C}:V\to k^n,\quad v\mapsto [v]_{\mathcal C}. \end{align*} Because $\mathcal B$ and $\mathcal C$ are ordered bases, every vector of $V$ has a unique coordinate column in each basis. Hence both coordinate maps are bijections. By definition of the change-of-basis matrices, for every $v\in V$, \begin{align*} [v]_{\mathcal C}=P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B} \end{align*} and \begin{align*} [v]_{\mathcal B}=P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}. \end{align*}[/step]

[guided]The notation $P_{\mathcal C \leftarrow \mathcal B}$ means that the matrix starts with coordinates in the basis $\mathcal B$ and produces coordinates in the basis $\mathcal C$. To make that precise, define \begin{align*} \operatorname{coord}_{\mathcal B}:V\to k^n,\quad v\mapsto [v]_{\mathcal B} \end{align*} and \begin{align*} \operatorname{coord}_{\mathcal C}:V\to k^n,\quad v\mapsto [v]_{\mathcal C}. \end{align*} Since $\mathcal B$ is an ordered basis, each $v\in V$ is written uniquely as a linear combination of $b_1,\ldots,b_n$, so $[v]_{\mathcal B}$ is well-defined and every column in $k^n$ occurs as $[v]_{\mathcal B}$ for exactly one $v$. Thus $\operatorname{coord}_{\mathcal B}$ is bijective. The same argument applies to $\operatorname{coord}_{\mathcal C}$. The defining equations for the two change-of-basis matrices are therefore \begin{align*} [v]_{\mathcal C}=P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B} \end{align*} and \begin{align*} [v]_{\mathcal B}=P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C} \end{align*} for every $v\in V$. These two equations express that the matrices perform opposite coordinate conversions.[/guided]

[step:Show that converting from $\mathcal B$ to $\mathcal C$ and back fixes every column] Let $I_n$ denote the $n\times n$ identity matrix over $k$. For arbitrary $v\in V$, substitute the defining equation for $[v]_{\mathcal C}$ into the reverse change-of-basis equation: \begin{align*} P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B}=P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}. \end{align*} Using the defining equation for $P_{\mathcal B \leftarrow \mathcal C}$, the right-hand side equals $[v]_{\mathcal B}$. Hence \begin{align*} P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B}=[v]_{\mathcal B} \end{align*} for every $v\in V$. Because $\operatorname{coord}_{\mathcal B}:V\to k^n$ is surjective, every $x\in k^n$ has the form $x=[v]_{\mathcal B}$ for some $v\in V$. Therefore \begin{align*} P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}x=x \end{align*} for every $x\in k^n$. Thus \begin{align*} P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}=I_n. \end{align*} [/step]

[step:Show that converting from $\mathcal C$ to $\mathcal B$ and back also fixes every column] For arbitrary $v\in V$, substitute the defining equation for $[v]_{\mathcal B}$ into the forward change-of-basis equation: \begin{align*} P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}=P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B}. \end{align*} Using the defining equation for $P_{\mathcal C \leftarrow \mathcal B}$, the right-hand side equals $[v]_{\mathcal C}$. Hence \begin{align*} P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}=[v]_{\mathcal C} \end{align*} for every $v\in V$. Because $\operatorname{coord}_{\mathcal C}:V\to k^n$ is surjective, every $y\in k^n$ has the form $y=[v]_{\mathcal C}$ for some $v\in V$. Therefore \begin{align*} P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}y=y \end{align*} for every $y\in k^n$. Thus \begin{align*} P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}=I_n. \end{align*} [/step]

[step:Identify the inverse matrix] The matrix $P_{\mathcal B \leftarrow \mathcal C}$ is both a left inverse and a right inverse of $P_{\mathcal C \leftarrow \mathcal B}$: \begin{align*} P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}=I_n \end{align*} and \begin{align*} P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}=I_n. \end{align*} Therefore $P_{\mathcal C \leftarrow \mathcal B}$ is invertible, and its inverse is \begin{align*} (P_{\mathcal C \leftarrow \mathcal B})^{-1}=P_{\mathcal B \leftarrow \mathcal C}. \end{align*} This is the desired conclusion. [/step]