[step:Expand $\Delta^{p^k} f(x)$ and reduce modulo $p$]The $N$-th iterate of the forward difference operator satisfies
\begin{align*}
\Delta^N f(x) = \sum_{i=0}^{N} (-1)^{N-i} \binom{N}{i} f(x + i)
\end{align*}
for any $N \geq 0$. Taking $N = p^k$:
\begin{align*}
\Delta^{p^k} f(x) = \sum_{i=0}^{p^k} (-1)^{p^k - i} \binom{p^k}{i} f(x + i).
\end{align*}
[claim:$p \mid \binom{p^k}{i}$ for all $0 < i < p^k$]
For $0 < i < p^k$, the binomial coefficient $\binom{p^k}{i} = \frac{p^k}{i} \binom{p^k - 1}{i - 1}$ is divisible by $p$. To see this, write $i = p^a m$ with $\gcd(m, p) = 1$ and $0 \leq a < k$. Then $v_p\left(\binom{p^k}{i}\right) = v_p(p^k!) - v_p(i!) - v_p((p^k - i)!)$. By Kummer's theorem, $v_p\left(\binom{p^k}{i}\right)$ equals the number of carries when adding $i$ and $p^k - i$ in base $p$. Since $0 < i < p^k$, the base-$p$ representations of $i$ and $p^k - i$ must produce at least one carry (the sum is $p^k = (1\underbrace{00\cdots0}_{k})_p$, which has a $1$ in a position where both $i$ and $p^k - i$ have $0$), so $v_p\left(\binom{p^k}{i}\right) \geq 1$.
[/claim]
[proof]
By Kummer's theorem, $v_p\left(\binom{m+n}{m}\right)$ equals the number of carries when adding $m$ and $n$ in base $p$. Here $m = i$ and $n = p^k - i$, so $m + n = p^k$. In base $p$, the number $p^k$ is $1$ followed by $k$ zeros. If $i$ has base-$p$ representation $(d_{k-1}, \ldots, d_1, d_0)$ (with $0 \leq d_j \leq p-1$), then $p^k - i$ has digits that are the $(p-1)$-complements of $(d_{k-1}, \ldots, d_0)$ with a final carry propagation. Since $0 < i < p^k$, at least one digit $d_j$ is nonzero, which forces at least one carry. Hence $v_p\left(\binom{p^k}{i}\right) \geq 1$.
[/proof]
Since $\left|\binom{p^k}{i}\right|_p \leq p^{-1}$ for $0 < i < p^k$ and $|f(x+i)|_p \leq \|f\| = 1$, each interior term satisfies
\begin{align*}
\left|(-1)^{p^k-i}\binom{p^k}{i}f(x+i)\right|_p \leq p^{-1}.
\end{align*}
The boundary terms are $i = 0$ (contributing $(-1)^{p^k}f(x)$) and $i = p^k$ (contributing $f(x + p^k)$). Therefore by the ultrametric inequality:
\begin{align*}
\left|\Delta^{p^k}f(x) - \left((-1)^{p^k}f(x) + f(x+p^k)\right)\right|_p \leq p^{-1}.
\end{align*}
Since $(-1)^{p^k} = -1$ for odd $p$, and $(-1)^{2^k} = 1 = -1$ in $\mathbb{Z}_2/2\mathbb{Z}_2$ (i.e., $|1 - (-1)|_2 = |2|_2 = 2^{-1} \leq p^{-1}$ when $p = 2$), in both cases:
\begin{align*}
\left|\Delta^{p^k}f(x) - (f(x+p^k) - f(x))\right|_p \leq p^{-1}.
\end{align*}[/step]