The neuropsychologist Donald Hebb postulated in 1949 how biological neurons learn:
“When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place on one or both cells such that A’s efficiency as one of the cells firing B, is increased.”
In more familiar terminology, that can be stated as the Hebbian Learning rule:
If two neurons on either side of a synapse (connection) are activated simultaneously (i.e. synchronously), then the strength of that synapse is selectively increased.
Mathematically, we can describe Hebbian learning as:
Here, η is a learning rate coefficient, and x are the outputs of the ith and jth elements.
Now, my question is, what do all these descriptions mean?
- Is Hebbian Learning applicable for single-neuron networks?
- What does it mean by "two neurons on either side of a synapse"?
- Why/when would two neurons activate simultaneously?
- What does they mean by