I have an environment, in which my agent learns according to PPO. The environment has a maximum of 80 actions, however not all of them are always allowed. My idea was to mask them, by setting the probabilities of the non valid actions to 0, and renormalizing the remaining actions. However, this would be not the predicted policy anymore, and thus the agent wouldn't act on policy. Is there a better way to mask PPO agents actions, or does it simply not constitute a big problem?