Say I have a game with 4 base actions [left, right, up, down] and then a value n, which determines how many times the chosen action is repeated.
For example, action = left, n = 3 -> go left 3 times. In this game $(left,1)*3 \neq (left,3)$ as negative reward is handed out at every single time step (this is for research purposes, so it cannot change).
I would like to test how a DDQN and a DQN algorithm are affected, as I increase the number of actions available (increase $n$).
My question is; Is there a smarter way to implement this, other than increasing the depth of the output layer? I.e
len(output_layer) = n?
I was thinking of whether or not, there was a way for a single neuron to determine n and then have 4 other neurons that determine the best action? Would this even have any positive effects? (such as less training time, faster computation, better generalization, etc.)
If yes, how would this typically be done?