From this video tutorial Vanishing Gradient Tutorial, the sigmoid function and the hyperbolic tangent can produce the vanishing gradient problem.

What other activation functions can lead to the vanishing gradient problem?

  • $\begingroup$ Possibly, the saturating activation functions (i.e. activation functions that squash their input to a finite interval). Both sigmoid and hyperbolic tangents are saturating activation functions. $\endgroup$ – nbro Mar 16 '20 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.