While reading about reinforcement learning, I have come acrossSutton and Barto define the following expression for expected rewards in terms of a summationstate–action–next-state reward function, the denominator of which I am not able to account for.
The formula given is:$r(s, a, s')$, as follows (equation 3.6, p. 49)
$$ r(s, a, s^{\prime}) \doteq \mathbb{E}\left[R_{t} \mid S_{t-1}=s, A_{t-1}=a, S_{t}=s^{\prime}\right]=\sum_{r \in \mathcal{R}} r \frac{p(s^{\prime}, r \mid s, a )}{\color{red}{p(s^{\prime} \mid s, a)}} $$
According to what I understand, the formula should have been correct withoutWhy is the denominatorterm (that I have highlighted). How is$p(s' \mid s, a)$ required in this formuladefinition? Shouldn't the correct formula be $\sum_{r \in \mathcal{R}} r p(s^{\prime}, r \mid s, a )$?
Source of image: Andrew G and Sutton's book on RL.