14
$\begingroup$

I've heard multiple times that "Neural Networks are the best approximation we have to model the human brain", and I think it is commonly known that Neural Networks are modelled after our brain.

I strongly suspect that this model has been simplified, but how much?

How much does, say, the vanilla NN differ from what we know about the human brain? Do we even know?

$\endgroup$
9
$\begingroup$

How Artificial Neural Networks (ANNs) are different from the Biological Neural Networks (BNNs) depends on what you are looking for. We all know that the ANNs are inspired by the Biological ones.
Structural differences:
enter image description here
In general, a neural network consists of four components:

  • neurons
  • topology: the connectivity path between neurons
  • weights
  • learning algorithm

In case of artificial neural network initial state and weights are assigned randomly. While for biological neural networks strengths of connections between neurons and structure of connections don't start as random. The initial state is genetically derived and is the byproduct of evolution.
In BNN learning comes from the interconnections between myriad neurons in the brain. These interconnections change configuration when the brain experiences new stimuli. The changes result in new connections, strengthening of existing connections and removal of old and unused ones.
ANNs are trained from scratch using fixed topology (remember topology changes in case of BNNs), which depends on the problem being solved. Current mechanism don't change the topology of the ANN and weights are randomly initialized and adjusted via an optimization algorithm.

Another contrast is in the number of neurons in the network. A typical ANN consists of hundreds, or maybe thousands, of neurons; the biological neural network of the human brain consists of billions. This number varies from animal to animal.animal.
You can find more here and here.

$\endgroup$
  • 1
    $\begingroup$ Are the connection strength genetically derived? Are you sure about that? $\endgroup$ – DuttaA Apr 9 '18 at 9:37
  • 2
    $\begingroup$ Initially, the connection strengths of a newborn child are determined genetically. After that they change mostly on the basis of external stimuli. $\endgroup$ – Ugnes Apr 9 '18 at 9:41
  • 2
    $\begingroup$ Artificial neural networks are not restricted to fixed topologies (see NEAT, TWEANN, etc.) $\endgroup$ – Andrew Butler Apr 9 '18 at 20:44
  • 1
    $\begingroup$ An ANN for image recognition does definitely have many more than just thousands of neurons. Many millions probably. $\endgroup$ – maaartinus Apr 10 '18 at 3:23
  • $\begingroup$ I wrote the answer based on the general ANNs found today, not considering the best ANNs. With the improvement in technology, ANNs are also improving; moving closer to BNNs. Thanks, AndrewButler and maaartinus to mention these. $\endgroup$ – Ugnes Apr 10 '18 at 5:42
4
$\begingroup$

They are not close, not anymore!

[Artificial] Neural Nets vaguely inspired by the connections we previously observed between the neurons of a brain. Initially, there probably was an intention to develop ANN to approximate biological brains. However, the modern working ANNs that we see their applications in various tasks are not designed to provide us a functional model of an animal brain. As far as I know, there is no study claiming they have found something new in a biological brain by looking into the connections and weight distributions of let's say a CNN or RNN model.

$\endgroup$
  • $\begingroup$ LOL. I agree. The one thing they have in common is this. They are circuits in the most abstract sense of the word. There may be one other thing. Nature tried a bunch of stuff and we emerged. Then we tried a bunch of stuff and XNNs emerged (where X is one of A, C, or N). Both nets are a result of a large number of failures. $\endgroup$ – FauChristian Jul 29 '18 at 10:43
3
$\begingroup$

The common statement that Artificial Neural Networks are inspired by the neural structure of brains is only partially true.

It is true that Norbert Wiener, Claude Shannon, John von Neuman, and others began the path toward practical AI by developing what they then called the electronic brain. It is also true

  • Artificial networks have functions called activations,
  • Are wired in many-to-many relationships like biological neurons, and
  • Are designed to learn an optimal behavior,

but that is the extent of the similarity. Cells in artificial networks such as MLPs (multilayer perceptrons) or RNN (Recurrent neural networks) are not like cells in brain networks.

The perceptron, the first software stab at arrays of things that activate, was not an array of neurons. It was the application of basic feedback involving gradients, which had been in common use in engineering ever since James Watt's centrifugal governor was mathematically modeled by Gauss. Successive approximation, a principle that had been in use for centuries, was employed to incrementally update an attenuation matrix. The matrix was multiplied by the vector feeding an array of identical activation functions to produce output. That's it.

The projection in a second dimension to a multi-layer topology was made possible by the realization that the Jacobian could be used to produce a corrective signal that, when distributed as negative feedback to the layers appropriately, could tune the attenuation matrix of a sequence of perceptrons and the network as a whole would converge upon satisfactory behavior. In the sequence of perceptrons, each element is called a layer. The feedback mechanism is now called back propagation.

The mathematics used to correct the network is called gradient descent because it is like a dehydrated blind man using the gradient of the terrain to find water, and the issues of doing that are similar too. He might find a local minima (low point) before he finds fresh water and converge on death rather than hydration.

The newer topologies are the additions of already existing convolution work used in digital image restoration, mail sorting, and graphics applications to create the CNN family of topologies and the ingenious use of what is like a chemical equilibrium from first year chemistry to combine optimization criteria creating the GAN family of topologies.

Deep is simply a synonym for numerous in most AI contexts. It sometimes infers complexity in the higher level topology (above the vector-matrix products, the activations, and the convolutions).

Active research is ongoing by those who are aware how different these deep networks are from what neural scientists have discovered decades ago in mammalian brain tissue. And there are more differentiators being discovered today as learning circuitry and neuro-chemistry in the brain is investigated from the genomic perspective.

  • Neural plasticity ... change in circuit topology due to dendrite and axiom growth, death, redirection, and other morphing
  • Topological complexity ... large number of axioms crisscross without interacting and are deliberately shielded from cross-talk (independent) most likely because it would be disadvantageous to let them connect [note 1]
  • Chemical signaling ... mammalian brains have dozens of neuro-transmitter and neuro-regulation compounds that have regional effects on circuitry [note 2]
  • Organelles ... living cells have many substructures and it is known that several types have complex relationships with signal transmission in neurons
  • Entirely different form of activation ... activations in common artificial neural nets are simply functions with ordinal scalars for both range and domain ... mammalian neurons operate as a function of both amplitude and relative temporal proximity of incoming signals [note 3]

[1] Topology is ironically both a subset of architecture (in the fields of building design, network provisioning, WWW analysis, and semantic networks), yet at the same time topology is, much more than architecture, at the radical center of both AI mathematics and effective actualization in control systems

[2] The role of chemistry may be essential to learning social and reproductive behavior that interrelates with DNA information propagation, linking in complex ways learning at the level of an ecosystem and the brain. Furthermore, long term and short term learning divides the brain's learning into two distinct capabilities too.

[3] The impact of the timing of incoming signals on biological neuron activation is understood to some degree, but it may impact much more than neuron output. It may impact placticity and chemistry too, and the organelles may play a role in that.

Summary

What machine learning libraries do is as much simulating the human brain as Barbie and Ken dolls simulate a real couple.

Nonetheless, remarkable things are arising in the field of deep learning, and it would not surprise me if autonomous vehicles become fully autonomous in our lifetimes. I would not recommend to any students to become a developer either. Computers will probably code much better than humans and orders of magnitude faster, and possibly soon. Some tasks are not of the kind that biology has evolved to do and computers can exceed human capabilities after only a few decades of research, eventually exceeding human performance by several orders of magnitude.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.