6

The most straightforward solution is to simply make every action "legal", but implementing a consistent, deterministic mapping from potentially illegal actions to different legal actions. Whenever the PPO implementation you are using selects an illegal action, you simply replace it with the legal action that it maps to. Your PPO algorithm can then still ...


2

Change the action space at each step, depending on the internal_state. I assume this is nonsense. Yes, this seems overkill and makes the problem unnecessarily complex, there could be other things you can do. Do nothing : let the model understand that choosing an unavailable action has no impact. While this will not harm your model negatively, in any way ...


1

Normally, the set of actions that the agent can execute does not change over time, but some actions can become impossible in different states (for example, not every move is possible in any position of the TicTacToe game). Take a look as example at pice of code https://github.com/haje01/gym-tictactoe/blob/master/examples/base_agent.py : ava_actions = env....


Only top voted, non community-wiki answers of a minimum length are eligible