2
$\begingroup$

I'm interested in building a (deep) RL agent for solving a continuous problem (which splits something into portions).

In all examples I've seen so far, e.g., solving the continuous lunar lander, always a $\tanh$ output layer activation was used, which produces values between $-1$ and $+1$.

Is this just because it fits the use case or is this a general rule for RL agents with continuous action spaces?

What if I just want values between $0$ and $1$? Could I simply use a $\operatorname{softmax}$ activation for my output layer?

$\endgroup$
1
$\begingroup$

the use of Tanh is purely because it fits the described problem (Especially for values that are min-max normalized). I have worked on couple of professional RL projects ( specifically with actions in the continuous space) and I did not use tanh at all. Hope that helped :)

$\endgroup$
  • 1
    $\begingroup$ What activation function did you use then? Are there rules of thumb which activation function to use for which use case? For example, in classification there's the rule to use sigmoid for binary classification and softmax for multi-class classification. $\endgroup$ – CGFoX Mar 26 at 8:37
  • 1
    $\begingroup$ There is no rule of thumb, it totally depends on your own problem. But as a good practice, you might want to use a function that bounds the values in a realistic range according to the actions that you want to predict. $\endgroup$ – SellaDev Mar 26 at 10:03
  • $\begingroup$ Ok, that's not the answer I was hoping for :D But thank you anyways! $\endgroup$ – CGFoX Mar 26 at 15:44
  • $\begingroup$ Sorry for the bad news, but if you want to excel in machine learning in general, there is no actual rule of thumb for everything (including sigmoid for binary classification, it is just widely used and not a rule of thumb ), you have to try to think freely and solve each problem on it's own. you are welcome. $\endgroup$ – SellaDev Mar 27 at 5:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.