How does DDPG algorithm know about my action mapping function?

Question

I am using DDPG to solve a RL problem. The action space is given by the Cartesian product $[0,20]^4\times[0,6]^4$. The actor is implemented as a deep neural network with an output dimension equals to $8$ with tanh activation.

So, given a state s, an action is given by a = actor(s) where a contains real numbers in [-1,1]. Next, I map this action a into a valid action valid_a that belongs to the action space $[0,20]^4\times[0,6]^4$. Than, I use valid_a to calculate the reward.

My question is: how does the DDPG algorithm know about this mapping that I am doing? In what part of the DDPG algorithm should I specify this mapping? Should I provide a bijective mapping to guarantee that the DDPG algorithm learns bad from good actions?

Answer 1

I would recommend doing is allowing your network to output any real number and then clipping the output. For instance, I was working with an agent that had to learn an angle between $[0, 2\pi]$ and $[0, 1]$. If the network outputted e.g. 10 in the first dimension then this would just be clipped to $2\pi$.

This way the agent only learns about actions within the action space and the weights of the network would eventually be adjusted to only output actions within this action space, provided that the boundaries aren't the optimal actions.