1
$\begingroup$

I am working on a deep Q-learning project. My project is different than normal deep Q-learning. The rewards of my neural network must be positive because I need their values to importance sample actions. I know that I can't use ReLU as the activation function of my neural network. So the only suitable functions which I know are sigmoid, softmax and exponential function. I tried working with sigmoid and softmax but they generate wrong results and the loss function diverges. There are two terminal states in my model. Their rewards are 1 and 0. All other states don't have any immediate rewards.

$\endgroup$
0
$\begingroup$

First of all: an activation function is usually placed after a linear operation and you can have a lot of them (maybe different) in your nn. That's why it would be better to say $\bf a$ activation function of the neural net and not $\bf the$ activation function. So if you meant by activation function the last operation which is going to make your outputs non-negative, then you're right, ReLU isn't a good choice. Usually when people need to output some positive values, they take exponent as the last operation. I used it several times and everything was just fine.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.