Skip to main content
Artificial Intelligence

Return to Answer

added 192 characters in body
Source Link
Peteris
Peteris
  • 893
  • 5
  • 8

Training

While "running" a neural network can be done with any activation functions, we usually want to train it - i.e., adjust its parameters so that the result would be closer to what we desire.

This is commonly done by Backpropagationbackpropagation and variations of gradient descent, which requires the existence of a gradient - i.e., requires activation function to be differentiable - because the. The adjustment of each parameter is calculated from the derivationgradient of the activation function(s) that this parameter affects, so if you cannot get a gradient, then this approach can't be used.

Training

While "running" a neural network can be done with any activation functions, we usually want to train it - i.e., adjust its parameters so that the result would be closer to what we desire.

This is commonly done by Backpropagation, which requires the activation function to be differentiable - because the adjustment of each parameter is calculated from the derivation of the activation function(s) that this parameter affects.

Training

While "running" a neural network can be done with any activation functions, we usually want to train it - i.e., adjust its parameters so that the result would be closer to what we desire.

This is commonly done by backpropagation and variations of gradient descent, which requires the existence of a gradient - i.e., requires activation function to be differentiable. The adjustment of each parameter is calculated from the gradient of the activation function(s) that this parameter affects, so if you cannot get a gradient, then this approach can't be used.

Source Link
Peteris
Peteris
  • 893
  • 5
  • 8

Training

While "running" a neural network can be done with any activation functions, we usually want to train it - i.e., adjust its parameters so that the result would be closer to what we desire.

This is commonly done by Backpropagation, which requires the activation function to be differentiable - because the adjustment of each parameter is calculated from the derivation of the activation function(s) that this parameter affects.