Are there any commonly used activation functions (e.g. that take values in $(0,.5)\cup (.5,1)$)? Preferably for classification?

Why? I was looking for commonly used activation functions on Google, and I noticed that all activation functions are continuous. However, I believe this is not needed in Hornik's paper.

When I did a bit of testing myself, with a discontinuous activation function on the MNIST dataset, the results were good. So I was curious if anyone else used this kind of activation function.

  • $\begingroup$ The example you provide is discontinuous in strict mathematical sense, but in an informal way not really discontinuous (as far as I understand). So what exactly do you mean by discontinuous? Also can you link this paper. $\endgroup$
    – user9947
    Commented Apr 1, 2020 at 1:30
  • $\begingroup$ I'm looking for a commonly used discontinuous bounded activation function. Somethign I can use for classification.. $\endgroup$
    – ABIM
    Commented Apr 1, 2020 at 6:49


You must log in to answer this question.