I've seen several people say that sigmoids are like a saturating firing rate of a neuron but I don't see how or why they interpret it as such. I especially don't see the relationship between a "rate" (so a number of something over time, I guess here it's the number that a neuron activates in a unit of time) and the sigmoid graph. For me it resembles more to the voltage output of an operational amplifier in some cases.

  • 1
    $\begingroup$ Sigmoid outputs a value between 0-1, this can be interpreted as the probability of the neuron ‘firing’. $\endgroup$
    – David
    Commented Mar 21, 2021 at 22:20
  • $\begingroup$ @DavidIreland that could be an interpretation. Like the probability of a neuron "firing" when the input is x is $\sigma (x)$ (where $\sigma$ denotes the sigmoid function). $\endgroup$
    – Daviiid
    Commented Mar 21, 2021 at 23:17
  • $\begingroup$ It would be nice if you could provide 1-2 references where people say "sigmoids are like a saturating firing rate of a neuron", just to have more context. $\endgroup$
    – nbro
    Commented Mar 22, 2021 at 9:26
  • $\begingroup$ @nbro You're right, sorry for this imprecision. So a friend of mine told me about this interpretation (it was given by his professor and he found it cool) and when I searched for it I found this post: datascience.stackexchange.com/questions/14349/… I agree it's not many people I just thought that since it's a professor who said it, he maybe have read/heard it somewhere else etc... ^^' $\endgroup$
    – Daviiid
    Commented Mar 22, 2021 at 11:54


You must log in to answer this question.

Browse other questions tagged .