I am training an RL agent (specifically using the PPO algorithm) on a game environment with 2 possible actions left or right.
The actions can be taken with varying "force"; e.g. go left 17% or go right 69.3%. Currently, I have the agent output 21 actions - 10 for left (in 10% increments), 10 for right in 10% increments and 1 for stay in place (do nothing). In other words, there is a direct 1-1 mapping in 10% increments between the agent output and the force the agent uses to move in the environment.
I am wondering, if instead of outputting 21 possible actions, I change the action space to a binary output and obtain the action probabilities. The probabilities will have the form, say, [70, 30]. That is, go left with 70% probability and go right with 30% probability. Then I take these probabilities and put them through a non-linearity that translates to the actual action force taken; e.g an output of 70% probability to go left, may in fact translate to moving left with 63.8% force.
The non linear translation is not directly observed by the agent but will determine the proceeding state, which is directly observed.
I don't fully understand what the implications of doing this will be. Is there any argument that this would increase performance (rewards) as the agent does not need to learn direct action mappings, rather just a binary probability output?