In Reinforcement Learning, policies are defined in terms of possible actions (see for instance page 58 of the book by Sutton et al.). So, is any action that an agent has in its repertoire always "possible", even when it has a policy-value of zero? Or are possible actions dependent on the state?
3
-
$\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$– Community BotSep 2, 2022 at 9:53
-
$\begingroup$ Suppose that the action space is defined as $\mathcal{A} = \{1, 2, 3\}$ and the state space is $\mathcal{S} = \{a, b, c\}$. If we can only take actions $1$ and $2$ in state $a$ then what this means is that any policy must have $p(3 | a) = 0$ by definition. Here I guess I am trying to emphasise that whilst action $3$ is still part of the action space for state $a$, it is not a possible action, and so any policy must assign 0 probability to selecting it. $\endgroup$– DavidSep 3, 2022 at 21:29
-
$\begingroup$ Clear. Thank you. $\endgroup$– Julius BaerMar 21 at 8:33
Add a comment
|