At the time when the basic building blocks of machine learning (the perceptron layer and the convolution kernel) were invented, the model of the neuron in the brain taught at the university level was simplistic.
Back when neurons were still just simple computers that electrically beeped untold bits to each other over cold axon wires, spikes were not seen as the hierarchical synthesis of every activity in the cell down to the molecular scale that we might say they are today. In other words, spikes were just a summary report of inputs to be integrated with the current state, and passed on. In comprehending the intimate relationships of mitochondria to spikes (and other molecular dignitaries like calcium) we might now more broadly interpret them as synced messages that a neuron sends to itself, and by implication its spatially extended inhabitants. Synapses weigh this information heavily but ultimately, but like the electoral college, fold in a heavy dose of local administration to their output. The sizes and positions within the cell to which mitochondria are deployed can not be idealized or anthropomorphized to be those metrics that the neuron decides are best for itself, but rather what is thermodynamically demanded.1
Notice the reference to summing in the first bolded phrase above. This is the astronomically oversimplified model of biology upon which contemporary machine learning was built. Of course ML has made progress and produced results. This question does not dismiss or criticize that but rather widen the ideology of what ML can become via a wider field of thought.
Notice the second two bolded phrases, both of which denote statefulness in the neurons. We see this in ML first as the parameters that attenuate the signals between arrays of artificial neurons in perceptrons and then, with back-propagation into deeper networks. We see this again as the trend in ML pushes toward embedded statefulness by integrating with object oriented models, the success of LSTM designs, the interrelationships of GAN designs, and the newer experimental attention based network strategies.
But does the achievement of higher level thought in machines, such as is needed to ...
- Fly a passenger jet safely under varying conditions,
- Drive a car in the city,
- Understand complex verbal instructions,
- Study and learn a topic,
- Provide thoughtful (not mechanical) responses, or
- Write a program to a given specification
... requiring from us a much more radical is the transition in thinking about what an artificial neuron should do?
Scientific research into brain structure, its complex chemistry, and the organelles inside brain neurons have revealed significant complexity. Performing a vector-matrix multiplication to apply learning parameters to the attenuation of signals between layers of activations is not nearly a simulation of a neuron. Artificial neurons are not very neuron-like, and the distinction is extreme.
A little study on the current state of the science of brain neuron structure and function reveals the likelihood that it would require a massive cluster of GPUs training for a month just to learn what a single neuron does.
Are artificial networks based on the perceptron design inherently limiting?
References
[1] Fast spiking axons take mitochondria for a ride, by John Hewitt, Medical Xpress, January 13, 2014, https://medicalxpress.com/news/2014-01-fast-spiking-axons-mitochondria.html