[guided]We first isolate the only analytic fact about $d_f$ that will be needed later. Define $M:=\mu(E)$. If $0\le a\le b$, then every point satisfying $f(x)>b$ also satisfies $f(x)>a$. Hence
\begin{align*}
\{x\in E:f(x)>b\}\subset \{x\in E:f(x)>a\}.
\end{align*}
Applying the measure $\mu$ to this inclusion gives
\begin{align*}
d_f(b)=\mu(\{x\in E:f(x)>b\})\le \mu(\{x\in E:f(x)>a\})=d_f(a).
\end{align*}
Thus $d_f$ is nonincreasing.
Now fix $t\ge0$. To prove right-continuity, we approach $t$ from above. Define a sequence
\begin{align*}
a_n:=t+\frac{1}{n}
\end{align*}
for $n\in\mathbb N$, and define measurable sets
\begin{align*}
A_n:=\{x\in E:f(x)>a_n\}.
\end{align*}
Since $a_n\downarrow t$, the condition $f(x)>a_n$ becomes weaker as $n$ increases, so $(A_n)_{n\ge1}$ is an increasing sequence of measurable sets. Its union is exactly the strict superlevel set at $t$:
\begin{align*}
\bigcup_{n=1}^{\infty}A_n=\{x\in E:f(x)>t\}.
\end{align*}
Indeed, if $f(x)>t$, then $f(x)>t+1/n$ for all sufficiently large $n$; the reverse inclusion follows from $a_n>t$ for every $n$.
Continuity from below for measures applies to the increasing sequence $(A_n)_{n\ge1}$ and gives
\begin{align*}
\mu\left(\bigcup_{n=1}^{\infty}A_n\right)=\lim_{n\to\infty}\mu(A_n).
\end{align*}
Substituting the definitions of $A_n$ and $d_f$, we obtain
\begin{align*}
d_f(t)=\mu(\{x\in E:f(x)>t\})=\lim_{n\to\infty}d_f\left(t+\frac{1}{n}\right).
\end{align*}
We now justify the last sentence with the squeeze argument. Let $u>t$. Choose $n\in\mathbb N$ such that $u<t+1/n$. Because $d_f$ is nonincreasing and $t<u<t+1/n$, the values of $d_f$ satisfy
\begin{align*}
d_f(t)\ge d_f(u)\ge d_f(t+1/n).
\end{align*}
Now let $\delta>0$. From the convergence already proved, choose $n\in\mathbb N$ such that
\begin{align*}
d_f(t)-d_f(t+1/n)<\delta.
\end{align*}
Then every $u\in(t,t+1/n)$ satisfies
\begin{align*}
0\le d_f(t)-d_f(u)\le d_f(t)-d_f(t+1/n)<\delta.
\end{align*}
This proves $d_f(u)\to d_f(t)$ as $u\downarrow t$, which is exactly right-continuity at $t$.
We also need one well-definedness fact for the generalized inverse defining $f^*$. Namely, for each $s\in(0,M)$, the set
\begin{align*}
\{a\in[0,\infty):d_f(a)\le s\}
\end{align*}
must be nonempty. To prove this, define
\begin{align*}
B_m:=\{x\in E:f(x)>m\}
\end{align*}
for $m\in\mathbb N$. These sets are measurable and decreasing in $m$. Since $f$ is finite-valued with codomain $[0,\infty)$, no point $x\in E$ can satisfy $f(x)>m$ for every $m\in\mathbb N$, so
\begin{align*}
\bigcap_{m=1}^{\infty}B_m=\varnothing.
\end{align*}
The hypothesis $\mu(E)<\infty$ gives $\mu(B_1)<\infty$, so continuity from above applies to the decreasing sequence $(B_m)_{m\ge1}$. Hence
\begin{align*}
\lim_{m\to\infty}d_f(m)=\lim_{m\to\infty}\mu(B_m)=\mu(\varnothing)=0.
\end{align*}
Therefore, if $s\in(0,M)$, we may choose $m\in\mathbb N$ with $d_f(m)\le s$. This proves that the set over which $f^*(s)$ takes its infimum is nonempty.[/guided]