Is there a method (not a table of recommendations!) that could tell me what activation function to choose if the outputs of the neural network have some interpretation? For example, these can be the mean of some normal distribution, probabilities in multinomial distribution, parameters of the exponential distribution, and so on.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ This is an old question, but I think you should clarify what you really mean by "interpretation". Do you mean that the outputs are just "the mean of some normal distribution, probabilities in multinomial distribution, parameters of the exponential distribution", i.e. related to distributions, or maybe something else? $\endgroup$– nbroNov 6, 2020 at 2:33
-
$\begingroup$ Yes, please, clarify. NNs are just powerful aproximators. You can interpret the network outputs as you want (and treat them accordingly in the loss_fn). The act_fn are more an architectural design question rather than a information coding/interpretation question. $\endgroup$– JVGDNov 6, 2020 at 10:30
Add a comment
|