Tag Info

My guess is that $r(s,a)$ is the constant so it can be moved out of the summation, leaving $r(s,a)\sum_{s'}P^{a}_{ss'} = r(s,a)$ Yes, this is the case. More specifically: $r(s,a)$ is the expected reward after taking action $a$ in state $s$. Reward may depend on the state arrived in, $s'$, but that is ignored in the equations. Reward may vary randomly, but ...