6
$\begingroup$

On the Wikipedia page we can read the basic structure of an artificial neuron (a model of biological neurons) which consist:

  • Dendrites - acts as the input vector,
  • Soma - acts as the summation function,
  • Axon - gets its signal from the summation behavior which occurs inside the soma.

I've checked Deep learning wiki page, but I couldn't find any references to dendrites, soma or axons.

So my question is, which type of artificial neural network implements or can mimic such model most closely?

$\endgroup$
  • 1
    $\begingroup$ Your mapping of dendrites, soma, and axon to the computational model is somewhat simplistic, but I understand where you are intending to go. Considerable evidence indicates that learning mechanisms are as much if not more dependent on the formation of new neurons and new dendrites and associated axon terminals than in activation threshold state changes. Several structures in the nuerofibrils have been identified as probably relate to memory and learning too. $\endgroup$ – FauChristian Aug 31 '17 at 9:53
2
$\begingroup$

Only a small portion of the habituation, sensitization, and classical conditioning behavior of neurons has been primitively simulated in ANN systems. Simulation of actin cytoskeletal machinery1 and other agents of neural plasticity, central to learning new domains, is in its beginnings2. As of this writing, the complexity of neuron activation dwarfs the models being used in working commercial ANN systems, but the research continues along multiple fronts.

  • The neuroscience of learning3,
  • Parallel hardware approaches that better support ANN simulation accuracy4, 5, and
  • Dynamic frameworks6

This list and the examples referenced in the superscripts, with links below, represent a tiny sample of the information available and the work in progress.

References

[1] Molecular Cell Biology. 4th edition.; Lodish H, Berk A, Zipursky SL, et al.; New York: W. H. Freeman; 2000.; Section 18.1 The Actin Cytoskeleton

[2] NEURON Software; Yale U

[3] Molecular Cell Biology. 4th edition.; Lodish H, Berk A, Zipursky SL, et al.; New York: W. H. Freeman; 2000.; Section 21.7 Learning and Memory

[4] Artificial Neural Networks on Massively Parallel Computer Hardware; Udo Seiffert; University of Magdeburg, Germany

[5] NeuroGrid; Stanford U

[6] Explanation of Dynamic Computational Graph frameworks

$\endgroup$
1
$\begingroup$

ANNs approximate biological neuronal networks. The approximation began with extreme simplicity in the early perceptron design. Spiking networks are examples of more accurate approximations. More accurate still, are complex simulations of neuron behavior that therefore necessitate significant computing resources.

If you are interested in a mathematical overview on analysis of biological neuron models I can recommend Dynamical Systems in Neuroscience by Eugene Izhikevich.

$\endgroup$
  • 1
    $\begingroup$ A spiking network is an ANN. You may wish to contrast neuron models to not mislead readers into thinking otherwise. $\endgroup$ – FauChristian Aug 31 '17 at 10:52
  • $\begingroup$ Yes, you are right. In that sense any simulation or even hardware model should be considered artificial. But if people say ANN they often mean the neural networks used in deep learning etc., i.e. without axons, dendrites, and even the time dimension, thus also non-spiking. $\endgroup$ – André Bergner Sep 4 '17 at 8:41
  • $\begingroup$ I completely agree, and that is why I have an objection to the second sentence. ... You may agree that, in a Venn diagram, ANN would be shown as a superset of spiking networks, if one wishes to preserve semantic accuracy. To let the acronym ANN evolve to mean, "All neural networks that are artificial, EXCLUDING some more advanced types of neuron models," would mirror in AI the horrible trend seen in other disciplines, where the meaning of a term diverges from the meaning of its individual words. ... I'm submitting an edit for peer review. I hope you don't mind. $\endgroup$ – FauChristian Sep 4 '17 at 20:43
  • $\begingroup$ Totally agree, I don't mind if you edit, please go ahead :) $\endgroup$ – André Bergner Sep 5 '17 at 8:13
0
$\begingroup$

ANN research does not try to model biological neurons, as the aim is to achieve better performance at prediction tasks. However, there is a body of literature in neuroscience that looks at Computational models of neurons. Neurons are complicated cells and our understanding of neurons is still not complete.

$\endgroup$
  • 2
    $\begingroup$ ANN research actually DOES try to model biological neurons. That's why we call them Artificial Neural Networks. It is just that the models are primitive in working systems because of limitations in computational capacity of hardware architectures in common production. The other two sentences are correct, but brief. $\endgroup$ – FauChristian Aug 31 '17 at 9:57
  • $\begingroup$ The distinction is if you're applying your model to machine learning or artificial intelligence. Machine Learning aims at performance and profit. Think big business, cognitive services etc. $\endgroup$ – Zakk Diaz Aug 31 '17 at 20:03
  • 1
    $\begingroup$ @ZakkDiaz, AI researchers were interested in both performance and profit before most of those writing machine learning code today graduated kindergarten. I grant you that, since ARPANET went public, the volume of unreliable and poorly organized data has grown to astronomical proportions and requires supercomputing to analyze, requiring larger investments and therefore multinational funding sources. To a large degree though, the trends in R&D and the associated terminology are largely independent from one another, the later being mostly branding, and both trends are somewhat arbitrary. $\endgroup$ – FauChristian Sep 3 '17 at 1:46
  • $\begingroup$ I can appreciate your comments, the technology its self is extremely old. You can find nearly identical principles in Control Systems (Automatic Control) which can date back to nearly 300 BC. This comes before the concept of a neuron even existed. I was only saying that in modern practices, Machine Learning and Artificial intelligence does have a application distinction. $\endgroup$ – Zakk Diaz Sep 5 '17 at 17:48
0
$\begingroup$

A computational model that attempts to closely mimic the human neural networks is Numenta's hierarchical temporal memory (which has not yet received much attention from the machine learning community). In their models, they explicitly model and implement dendrites and other biological concepts.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.