8
$\begingroup$

I was reading a paper on alternatives to backpropagation as a learning algorithm in neural networks. In this paper, the author talks about the disadvantages of backpropagation, and one of the disadvantages stated is that backpropagation requires symmetric weights and that's why it's not biologically plausible.

What do symmetric weights mean and how does it make backpropagation biologically implausible?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
8
$\begingroup$

"Symmetric weights" means that the same weight value associated to a pair of nodes must be used during the forwards and backwards steps.

The reason it makes back propagation biologically impossible in its naive formulation is that neurons fire electrical signals in only one direction, from the dendrite through the axon to other dendrites of other neurons. They do receive of course "backwards" feedback, but by other means, e.g. chemical neurotransmitters or other signals from other neurons, but these signals are very likely not of the same intensity as the signal emitted by the neurons themselves (i.e. no symmetry).

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Thanks! Makes sense to me now. $\endgroup$
    – 0jas
    May 8, 2022 at 18:21
  • 1
    $\begingroup$ I am not a neuroscientist, but you should probably note that this is an oversimplification and that not everything is understood about brains and our human brain. Moreover, I would also like to see some literature that supports your claims, especially this "these signals are very much likely not of the same intensity of the signal emitted by the neurons themselves". $\endgroup$
    – nbro
    May 8, 2022 at 20:41
  • $\begingroup$ @nbro Which part are you calling an oversimplification? The obvious (to me) part is already highlighted: “in its naive formulation”. $\endgroup$
    – wizzwizz4
    May 9, 2022 at 11:41
  • $\begingroup$ @wizzwizz4 The “in its naive formulation" part refers to back-propagation in ANNs, as far as I interpret that sentence, i.e. "in the naive formulation of back-propagation in ANNs". I was referring to his description of how (biological) neural circuits work - it's an oversimplification. $\endgroup$
    – nbro
    May 9, 2022 at 12:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .