1
$\begingroup$

I have a doubt about how clipping affects the training of the RL agents.

In particular, I have come across a code for training DDPG agents, the pseudo-code is the following:

1  for i in training iterations
2      action = clip(ddpg.prediction(state) * a + b, x, y)
3      state, reward = environment(action)
4      store action, state and reward
5      if the number of experiences is larger than L:
6          update the parameters of the agent

In this case, the actor NN that predicts the DDPG has a $\tanh$ activation in the output layer.

My question is, could we add the clipping in the output layer of the actor (changing $\tanh(x)$ by $\operatorname{clip}(a\cdot \tanh(x)+b, x, y$) in the training loop? Would the training work in that case?

Improve this question
$\endgroup$
2
  • $\begingroup$ Hello. It may be a good idea to provide a link to the source where you took this pseudocode from, for people that want to have more context. $\endgroup$
    – nbro
    Commented Sep 11, 2021 at 13:05
  • 1
    $\begingroup$ The code is based on the trained loop used by this project, you can see it on the link. I have written it using pseudo code because the question is related to the training of the agent rather than with the code $\endgroup$
    – Leibniz
    Commented Sep 11, 2021 at 14:05

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .