[proofplan]
We prove both implications directly from the least-upper-bound definition of $\sup A$. If $s=\sup A$, then $s$ is an upper bound, and every number strictly below $s$ fails to be an upper bound; applying this to $s-\varepsilon$ gives an element of $A$ within $\varepsilon$ below $s$. Conversely, if $s$ is an upper bound and every interval $(s-\varepsilon,s]$ meets $A$, then every number $t<s$ fails to be an upper bound by choosing $\varepsilon=s-t$. Hence $s$ is the least upper bound of $A$.
[/proofplan]
[step:Derive the epsilon witnesses from the least-upper-bound property]
Assume $s=\sup A$. By the definition of supremum, $s$ is an upper bound for $A$, so for every $a \in A$ one has $a \leq s$.
Let $\varepsilon \in \mathbb{R}$ satisfy $\varepsilon>0$. Since $\varepsilon>0$, we have
\begin{align*}
s-\varepsilon < s.
\end{align*}
Because $s$ is the least upper bound of $A$, no real number strictly smaller than $s$ is an upper bound for $A$. Therefore $s-\varepsilon$ is not an upper bound for $A$. By the negation of the definition of upper bound, there exists $a \in A$ such that
\begin{align*}
s-\varepsilon < a.
\end{align*}
Since $s$ is an upper bound for $A$ and $a \in A$, we also have $a \leq s$. Thus
\begin{align*}
s-\varepsilon < a \leq s.
\end{align*}
Since $\varepsilon>0$ was arbitrary, the required epsilon condition holds.
[guided]
Assume $s=\sup A$. The meaning of this statement is twofold: first, $s$ is an upper bound for $A$; second, $s$ is the least among all upper bounds of $A$. From the upper-bound part, every element $a \in A$ satisfies
\begin{align*}
a \leq s.
\end{align*}
Now fix an arbitrary real number $\varepsilon>0$. We want to find an element of $A$ that lies above $s-\varepsilon$. The reason to look at $s-\varepsilon$ is that it is strictly smaller than $s$:
\begin{align*}
s-\varepsilon < s.
\end{align*}
Since $s$ is the least upper bound, any real number strictly below $s$ cannot itself be an upper bound for $A$. Therefore $s-\varepsilon$ is not an upper bound for $A$.
Unpacking the negation of upper boundedness at the specific number $s-\varepsilon$, there must exist some $a \in A$ such that
\begin{align*}
s-\varepsilon < a.
\end{align*}
The same element $a$ also satisfies $a \leq s$, because $s$ is an upper bound for all of $A$. Combining the two inequalities gives
\begin{align*}
s-\varepsilon < a \leq s.
\end{align*}
Since the choice of $\varepsilon>0$ was arbitrary, this proves the epsilon condition.
[/guided]
[/step]
[step:Use the epsilon witnesses to prove leastness of the upper bound]
Assume that $s$ is an upper bound for $A$ and that for every $\varepsilon>0$ there exists $a \in A$ such that
\begin{align*}
s-\varepsilon < a \leq s.
\end{align*}
To prove $s=\sup A$, it remains to prove that no real number strictly smaller than $s$ is an upper bound for $A$.
Let $t \in \mathbb{R}$ satisfy $t<s$. Define $\varepsilon := s-t$. Then $\varepsilon>0$. By the assumed epsilon condition, there exists $a \in A$ such that
\begin{align*}
s-\varepsilon < a \leq s.
\end{align*}
Since $\varepsilon=s-t$, we have $s-\varepsilon=t$, and hence
\begin{align*}
t<a.
\end{align*}
Thus $t$ is not an upper bound for $A$. Since every $t<s$ fails to be an upper bound, while $s$ is an upper bound for $A$, $s$ is the least upper bound of $A$. Therefore
\begin{align*}
s=\sup A.
\end{align*}
[/step]