Questions tagged [activation-function]

For questions related to the selection of and theory behind specific activation functions used in artificial networks.

Filter by
Sorted by
Tagged with
21
votes
6answers
11k views

What is the purpose of an activation function in neural networks?

It is said that activation functions in neural networks help introduce non-linearity. What does this mean? What does non-linearity mean in this context? How does the introduction of this non-...
15
votes
3answers
6k views

How to choose an activation function?

I choose the activation function for the output layer depending on the output that I need and the properties of the activation function that I know. For example, I choose the sigmoid function when I'm ...
9
votes
5answers
3k views

Why do activation functions need to be differentiable in the context of neural networks?

Why should an activation function of a neural network be differentiable? Is it strictly necessary or is it just advantageous?
3
votes
1answer
342 views

What is a “logit probability”?

DeepMind's paper "Mastering the game of Go without human knowledge" states in its "Methods" section on its "Neural network architecture" that the output layer of AlphaGo Zero's policy head is "A fully ...
2
votes
0answers
121 views

Non-monotonic activation function and XOR problem with perceptron

I already post the question on Stackoverflow : https://stackoverflow.com/questions/53785922/is-the-use-of-a-non-monotonic-activation-function-is-a-correct-and-viable-solut?noredirect=1#...