I have heard that sigmoid activation functions should not be used on neural networks with many hidden layers as the gradients tend to vanish in deep networks.

When should each of the common activation functions be used, and why?

  • ReLu
  • Sigmoid
  • Softmax
  • Leaky ReLu
  • TanH

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.