Do neural networks, trained with backpropagation algorithm, exploit the concept of synaptic plasticity?

Question

Is there some of Hebb's rule behind the concept of backpropagation learning rule of a simple supervised neural network, that for example is trained for classification task ?

I was reading about the concept of synaptic plasticity that is explained in simple words here https://qbi.uq.edu.au/:

Plasticity is the ability of the brain to change and adapt to new information. Synaptic plasticity is change that occurs at synapses, the junctions between neurons that allow them to communicate. The idea that synapses could change, and that this change depended on how active or inactive they were, was first proposed in the 1949 by Canadian psychologist Donald Hebb.

And more in particular I found in Wikipedia that:

Hebbian theory is a neuroscientific theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptation of brain neurons during the learning process.

In an artificial neural network, synaptic efficacy (that is the strength of communication between neurons, from https://www.frontiersin.org) depends on weights that are associated to connections between neurons (from what I understood reading the beginning of paragraph 44.3 The basic unit — the neuron in Handbook of Chemometrics and Qualimetrics: Part B https://www.sciencedirect.com).

Thus, when we train our supervised classifier so that it updates weights thanks to backpropagation algorithm (whose philosophy is the changing of weights to minimize an error function that characterizes the comparison between the network output and the ground truth), are we applying a sort of Hebb's Rule and, more properly, the concept of synaptic plasticity since weights change during the learning process?

Or Hebb's rule means only "Fire together, wire together" as I understand by reading the answer to this SE post How do you explain Hebbian Learning in an intuitive way?.

I read also this other SE answer to the post Is there a Hebb neural network? in which it is clearly explained that neural networks (or models) that can learn in a Hebbian fashion are different from those based on backpropagation algorithm.

But if synaptic plasticity means a changing in the connections between neurons (synapses), that in artifical neural networks means a changing in the weights, during a learning process, can I say that also neural networks based on back-propagation algorithms exploit the concept of synaptic plasticity (and so the Hebb's rule) ?

Answer 1

Even when it is always said that "neural network cells" are "inspired" in biological neurons, there are critical differences that made this similitude only an "inspiration". Thus, comparison of a CNN or any other kind of NN with brain is always a fuzzy comparison.

The biggest difference is probably the learning capacity: the human brain learns by itself, while the neural network needs an external system (the back-propagation algorithm) that feeds the NN with the network parameters (the knowledge).

In other words: a neural network, understood as the set of nodes that execute a basic function (lets say weighted addition and activation function) is absolutely unable to learn anything by itself. The knowledge is written in the neural network as a set of weights and other parameters. This is not an intelligent process, as it is not load a program in a conventional computer. The system who learns is the system who runs the back-propagation algorithm and this system is not a neural net.

In comparison, the human mind, understood as a network of cells,is able to learn by itself from the experience.

Thus, when we train our supervised classifier so that it updates weights thanks to backpropagation algorithm (whose philosophy is the changing of weights to minimize an error function that characterizes the comparison between the network output and the ground truth), are we applying a sort of Hebb's Rule and, more properly, the concept of synaptic plasticity since weights change during the learning process?

Absolutely no.

A learning process based in Hebb's rule will:

• not need an external system to run the back-propagation algorithm
• increase the weight between two cells when an artificial neuron and the next one triggers more or less at same time.

This is not the method of the back-propagation algorithm:

• the back-propagation algorithm doesn't execute in the neural net
• the back-propagation algorithm compares expected results and network ones and, using math calculus, modifies the network weights.

Nowadays, I do not know of any system based on biological neuron behavior and able to learn. Just a few models as been able to learn the eigenvector of a basic process. Even worst, no body as been able to replicate basic behavior of well know biological brains as the one of c-elegans worm, a brain that is completely mapped in number of neurons and synapses between them.