[proofplan]
We use the defining property of change-of-basis matrices: $P_{\mathcal C \leftarrow \mathcal B}$ converts $\mathcal B$-coordinate columns into $\mathcal C$-coordinate columns, while $P_{\mathcal B \leftarrow \mathcal C}$ converts in the reverse direction. Composing these two conversions sends every coordinate column back to itself. Since coordinate columns in an ordered basis range over all of $k^n$, both products of the two matrices are the identity matrix, which proves invertibility and identifies the inverse.
[/proofplan]
[step:Declare the coordinate maps and the two change-of-basis matrices]
For each ordered basis, define the coordinate maps
\begin{align*}
\operatorname{coord}_{\mathcal B}:V\to k^n,\quad v\mapsto [v]_{\mathcal B}
\end{align*}
and
\begin{align*}
\operatorname{coord}_{\mathcal C}:V\to k^n,\quad v\mapsto [v]_{\mathcal C}.
\end{align*}
Because $\mathcal B$ and $\mathcal C$ are ordered bases, every vector of $V$ has a unique coordinate column in each basis. Hence both coordinate maps are bijections.
By definition of the change-of-basis matrices, for every $v\in V$,
\begin{align*}
[v]_{\mathcal C}=P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B}
\end{align*}
and
\begin{align*}
[v]_{\mathcal B}=P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}.
\end{align*}
[guided]
The notation $P_{\mathcal C \leftarrow \mathcal B}$ means that the matrix starts with coordinates in the basis $\mathcal B$ and produces coordinates in the basis $\mathcal C$. To make that precise, define
\begin{align*}
\operatorname{coord}_{\mathcal B}:V\to k^n,\quad v\mapsto [v]_{\mathcal B}
\end{align*}
and
\begin{align*}
\operatorname{coord}_{\mathcal C}:V\to k^n,\quad v\mapsto [v]_{\mathcal C}.
\end{align*}
Since $\mathcal B$ is an ordered basis, each $v\in V$ is written uniquely as a linear combination of $b_1,\ldots,b_n$, so $[v]_{\mathcal B}$ is well-defined and every column in $k^n$ occurs as $[v]_{\mathcal B}$ for exactly one $v$. Thus $\operatorname{coord}_{\mathcal B}$ is bijective. The same argument applies to $\operatorname{coord}_{\mathcal C}$.
The defining equations for the two change-of-basis matrices are therefore
\begin{align*}
[v]_{\mathcal C}=P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B}
\end{align*}
and
\begin{align*}
[v]_{\mathcal B}=P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}
\end{align*}
for every $v\in V$. These two equations express that the matrices perform opposite coordinate conversions.
[/guided]
[/step]
[step:Show that converting from $\mathcal B$ to $\mathcal C$ and back fixes every column]
Let $I_n$ denote the $n\times n$ identity matrix over $k$. For arbitrary $v\in V$, substitute the defining equation for $[v]_{\mathcal C}$ into the reverse change-of-basis equation:
\begin{align*}
P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B}=P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}.
\end{align*}
Using the defining equation for $P_{\mathcal B \leftarrow \mathcal C}$, the right-hand side equals $[v]_{\mathcal B}$. Hence
\begin{align*}
P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B}=[v]_{\mathcal B}
\end{align*}
for every $v\in V$.
Because $\operatorname{coord}_{\mathcal B}:V\to k^n$ is surjective, every $x\in k^n$ has the form $x=[v]_{\mathcal B}$ for some $v\in V$. Therefore
\begin{align*}
P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}x=x
\end{align*}
for every $x\in k^n$. Thus
\begin{align*}
P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}=I_n.
\end{align*}
[/step]
[step:Show that converting from $\mathcal C$ to $\mathcal B$ and back also fixes every column]
For arbitrary $v\in V$, substitute the defining equation for $[v]_{\mathcal B}$ into the forward change-of-basis equation:
\begin{align*}
P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}=P_{\mathcal C \leftarrow \mathcal B}[v]_{\mathcal B}.
\end{align*}
Using the defining equation for $P_{\mathcal C \leftarrow \mathcal B}$, the right-hand side equals $[v]_{\mathcal C}$. Hence
\begin{align*}
P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}[v]_{\mathcal C}=[v]_{\mathcal C}
\end{align*}
for every $v\in V$.
Because $\operatorname{coord}_{\mathcal C}:V\to k^n$ is surjective, every $y\in k^n$ has the form $y=[v]_{\mathcal C}$ for some $v\in V$. Therefore
\begin{align*}
P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}y=y
\end{align*}
for every $y\in k^n$. Thus
\begin{align*}
P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}=I_n.
\end{align*}
[/step]
[step:Identify the inverse matrix]
The matrix $P_{\mathcal B \leftarrow \mathcal C}$ is both a left inverse and a right inverse of $P_{\mathcal C \leftarrow \mathcal B}$:
\begin{align*}
P_{\mathcal B \leftarrow \mathcal C}P_{\mathcal C \leftarrow \mathcal B}=I_n
\end{align*}
and
\begin{align*}
P_{\mathcal C \leftarrow \mathcal B}P_{\mathcal B \leftarrow \mathcal C}=I_n.
\end{align*}
Therefore $P_{\mathcal C \leftarrow \mathcal B}$ is invertible, and its inverse is
\begin{align*}
(P_{\mathcal C \leftarrow \mathcal B})^{-1}=P_{\mathcal B \leftarrow \mathcal C}.
\end{align*}
This is the desired conclusion.
[/step]
