4
$\begingroup$

I am not asking what activation function is better. I want to know what activation functions are more used in research or deployment. Also, are they used in combination? E.g., ReLU, ELUs, etc. I'd appreciate any statistics or insight on this.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Currently, both ReLU and ELUs are the most popular activation functions (AF) used in neural nets (NNs). This is because they eliminate the vanishing gradient problem that causes major problems in the training process and degrades the accuracy and performance of NN models.

Also, these AFs, more specifically ReLU, are very fast learning AF which makes them even more useful in research.

However, depending on the type of NN you working on, it's always a good practice to pay some attention to new studies.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Maybe you should provide some references that show that ReLU and ELU are the most widely used AFs. If there are some statistics, that would even be better. $\endgroup$
    – nbro
    Jul 27, 2020 at 13:24
  • 1
    $\begingroup$ @nbro I will have a look and update any answer I provide over time. I haven't found any good source to provide stat about it. Please let me know if you know any. Thanks for the suggestion and also for keeping this community well. $\endgroup$
    – pedrum
    Jul 28, 2020 at 13:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .