I am quite new to RL. I would like to know why a state-dependent reward function R(s) is more restrictive than a state-action-dependent reward function R(s, a)? And why is it that a policy can be optimal for the latter R(s, a) but not for the former(R(s).
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ Does this answer your question? How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent? $\endgroup$– nbroOct 3 at 13:09
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
you have in front of you 10 slot machines
you can play with any of them, and each of them have a specific winrate (reward function)
the only state of this MDP is the initial state, the one where you are in front of these slot machines
how do you model the reward?
a. $R(s)$: well, the state is only one, so the distribution of the rewards does not depend on the action you take/ slot machine you play
b. $R(s,a)$: well, now you have actions, so you can learn statistics about $R(s,a)$ (aka statistics about each slot machine), and thus take actions according to these statistics (policy)
