4
$\begingroup$

I have come to notice that the most commonly used activation functions are continuous. Is there any specific reason behind this? Results such as this paper have worked on training networks with discontinuous activations yet this does not seem to have taken off. Does anybody have insight into why this happens, or better yet an article talking about this?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Here and here are two related questions. $\endgroup$
    – nbro
    Jan 28, 2021 at 0:25
  • 1
    $\begingroup$ I had seen both before posting but none of them discusses why they are not wildly used or why most activations are continuous. $\endgroup$
    – AIM
    Jan 28, 2021 at 13:58
  • 2
    $\begingroup$ Hi @BIM, check this out, has some interesting thoughts. It's about step-function though. $\endgroup$
    – mark mark
    Jan 28, 2021 at 15:27

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .