In these slides, it is written
\begin{align} \left\|T^{\pi} V-T^{\pi} U\right\|_{\infty} & \leq \gamma\|V-U\|_{\infty} \tag{9} \label{9} \\ \|T V-T U\|_{\infty} & \leq \gamma\|V-U\|_{\infty} \tag{10} \label{10} \end{align} where
- $F$ is the space of functions on domain $\mathbb{S}$.
- $T^{\pi}: \mathbb{F} \mapsto \mathbb{F}$ is the Bellman policy operator
- $T: \mathbb{F} \mapsto \mathbb{F}$ is the Bellman optimality operator
In slide 19, they say that equality $9$ follows from
\begin{align} {\scriptsize \left\| T^{\pi} V-T^{\pi} U \right\|_{\infty} = \max_{s} \gamma \sum_{s^{\prime}} \operatorname{Pr} \left( s^{\prime} \mid s, \pi(s) \right) \left| V\left(s^{\prime}\right) - U \left(s^{\prime}\right) \right| \\ \leq \gamma \left(\sum \operatorname{Pr} \left(s^{\prime} \mid s, \pi(s)\right)\right) \max _{s^{\prime}}\left|V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right| \\ \leq \gamma\|U-V\|_{\infty} } \end{align}
Why is that? Can someone explain to me this derivation?
They also write that inequality \ref{10} follows from
\begin{align} {\scriptsize \|T V-T U\|_{\infty} = \max_{s} \left| \max_{a} \left\{ R(s, a) + \gamma \sum_{s^{\prime}} \operatorname{Pr} \left( s^{\prime} \mid s, a \right) V \left( s^{\prime} \right) \right\} -\max_{a} \left\{R(s, a)+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right) U\left(s^{\prime}\right)\right\} \right| \\ \leq \max _{s, a}\left|R(s, a)+\gamma \sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right) V\left(s^{\prime}\right) -R(s, a)-\gamma \sum \operatorname{Pr}\left(s^{\prime} \mid s, a\right) V\left(s^{\prime}\right) \right| \\ = \gamma \max _{s, a}\left|\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right)\left(V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right)\right| \\ \leq \gamma\left(\sum_{s^{\prime}} \operatorname{Pr}\left(s^{\prime} \mid s, a\right)\right) \max _{s^{\prime}}\left|\left(V\left(s^{\prime}\right)-U\left(s^{\prime}\right)\right)\right| \\ \leq \gamma\|V-U\|_{\infty} } \end{align}
Can someone explain to me also this derivation?