Hot answers tagged


Probably not worth the hassle. I guess that simply nobody cares. I'm all for having my function be differentiable at all points, but I don't think that having a single jump in the derivative matters much. The neuron basically either repeats its input (x>0) or outputs nearly nothing (x<0) and the point in between is not that important. Replacing ELU(x) =...

Only top voted, non community-wiki answers of a minimum length are eligible