I've been reading about different types of neurons that the traditional linear one. One example I came across is the Sigma-Pi neuron, where the activation function includes higher order terms, such as

<span class=$f$" />

where the activation has the additional higher order term on the right, as well as the standard term on the left. What are the advantages and disadvantages of using neurons like that? Why do we tend not to use them?

  • $\begingroup$ Can you please 1. put your specific question in the title, 2. format your question with mathjax, 3. provide a link to a reference that talks about this "sigma-pi neuron" (for people that are not familiar with it)? $\endgroup$
    – nbro
    Commented Mar 27, 2022 at 21:06
  • $\begingroup$ My question isn't specific to the sigma-pi neural network, but here is a link to that anyway: repository.up.ac.za/bitstream/handle/2263/29715/… The question at heart is fairly straightforward: why are we using linear activations of the form wX+c and not higher order activations? What advantages and disadvantages are there? $\endgroup$ Commented Mar 28, 2022 at 13:07


You must log in to answer this question.