7
$\begingroup$

So this is an introductory question. Whenever I read any book about Neural Nets or Machine Learning, their introductory chapter says that we haven't been able to replicate the brain's power due to its massive parallelism.

Now, in the modern times transistors have been reduced to the size of nano-meters, much smaller than the nerve cell. Also we can easily build very large supercomputers.

  • Computers have much larger memories than brain.
  • Can communicate faster than brain (clock pulse in nanoseconds).
  • Can be of arbitrarily large size.

So my question is why cannot we replicate the brain's parallelism if not its information processing ability (since brain is still not well understood) even with such advanced technology? What exactly is the obstacle we are facing?

$\endgroup$
  • $\begingroup$ Am not fully qualified neuroscientists,so It might also be interesting to look into some papers discussing multitasking, I'm quite certain plenty of research has been done related to that." what makes animal brain so special? do some reading about the Selfawareness(consciousness). $\endgroup$ – quintumnia Feb 6 '18 at 19:03
5
$\begingroup$

One probable hardware limiting factor is internal bandwidth. A human brain has 10^15 synapses. Even if each is only exchanging a few bits of information per second, that's on the order of 10^15 bytes/sec internal bandwidth. A fast GPU (like those used to train neural networks) might approach 10^11 bytes/sec of internal bandwidth. You could network 10,000 of these together to get something close to the total internal bandwidth of the human brain, but the interconnects between the nodes would be relatively slow, and would bottleneck the flow of information between different parts of the "brain."

Another limitation might be raw processing power. A modern GPU has maybe 5,000 math units. Each unit has a cycle time of ~1 ns, and might require ~1000 cycles to do the equivalent processing work one neuron does in ~ 1/10 second (this value is totally pulled from the air; we don't really know the most efficient way to match brain processing in silicon). So a single GPU might be able to match 5 x 10^8 neurons in real time. You would optimally need 200 of them to match the processing power of the brain. This back-of-the-envelope calculation shows that internal bandwidth is probably a more severe constraint.

$\endgroup$
  • $\begingroup$ According to my analysis,this answer is right fit to this question; what is the difference between the multi-core processors computers have and the neuro networks that exist in the human brain? I also do think the OP should revise the question, in order to avoid cross posting from psychology community. $\endgroup$ – quintumnia Feb 6 '18 at 19:25
  • $\begingroup$ @quintumnia you can edit the question as you see fit $\endgroup$ – DuttaA Feb 7 '18 at 6:33
  • $\begingroup$ I'm sorry, but this answer fails in some common and very old errors, present in AI from Turing times, nowadays mainly discarded. $\endgroup$ – pasaba por aqui Feb 9 '18 at 20:00
  • $\begingroup$ This is mostly fair, but not entirely so. A large GPU can have 10^10 transistors flipping at 10^11 Hz, versus 10^15 synapses flipping at 10^2 Hz. The inefficiencies you state are largely structural; it seems ridiculous to assume that a hypothetical perfect transistor layout would get multiple orders of magnitude less than 0.0001x the utility from a transistor flip than from a neuron activation. $\endgroup$ – Veedrac Feb 12 '18 at 16:40
  • 1
    $\begingroup$ @Veedrac - a transistor has two binary inputs; a neuron has thousands of analog inputs. If you look at serious neuron-simulation code, I was being overly generous to the silicon. Even if you simplify a neuron to a cell in a neural network, calculating the weighted output for one cycle is going to take a lot of transistor switches. $\endgroup$ – antlersoft Feb 12 '18 at 20:46
2
$\begingroup$

This has been my field of research. I've seen the previous answers that suggest that we don't have sufficient computational power, but this is not entirely true.

The computational estimate for the human brain ranges from 10 petaFLOPS (1 x 10^16) to 1 exaFLOPS (1 x 10^18). Let's use the most conservative number. The TaihuLight can do 90 petaFLOPS which is 9 x 10^16.

We see that the human brain is perhaps 11x more powerful. So, if the computational theory of mind were true then TaiHuLight should be able to match the reasoning ability of an animal about 1/11th as intelligent.

If we look at a neural cortex list, the squirrel monkey has about 1/12th the number of neurons in its cerebral cortex as a human. With AI, we cannot match the reasoning ability of a squirrel monkey.

A dog has about 1/30th the number of neurons. With AI, we cannot match the reasoning ability of a dog.

A brown rat has about 1/500th the number of neurons. With AI, we cannot match the reasoning ability of a rat.

This gets us down to 2 petaFLOPS or 2,000 teraFLOPS. There are 67 supercomputers worldwide that should be capable of matching this.

A mouse has half the number of neurons as a brown rat. There are 190 supercomputers that should be able to match its reasoning ability.

A frog or non-schooling fish is about 1/5th of this. All of the top 500 supercomputers are 2.5x as powerful as this. Yet, none is capable of matching these animals.

What exactly is the obstacle we are facing?

The problem is that a cognitive system cannot be defined using only Church-Turing. AI should be capable of matching non-cognitive animals like arthropods, round worms, and flat worms but not larger fish or most reptiles.

Edit: I guess I need to give more concrete examples. The NEST system has demonstrated 1 second of operation of 520 million neurons and 5.8 trillion synapses in 5.2 minutes on the 5 petaFLOPS BlueGene/Q. The current thinking is that if they could scale the system by 200 to an exaFLOPS then they could simulate the human cerebral cortex at the same 1/300th normal speed. This might sound reasonable but it doesn't actually make sense.

A mouse has 1/1000th as many neurons as a human cortex. So This same system should be capable today of simulating a mouse brain at 1/60th normal speed. So, why aren't they doing it?

$\endgroup$
  • $\begingroup$ But you are forgetting the bandwidth part $\endgroup$ – DuttaA Apr 3 '18 at 6:50
  • $\begingroup$ The bandwidth of what? $\endgroup$ – scientious Apr 3 '18 at 6:58
  • $\begingroup$ Bandwidth roughly means the amount of information that can be passed from one section to another...brain probably has a huge bandwidth due to the numerous branch in the dendrons $\endgroup$ – DuttaA Apr 3 '18 at 16:01
  • $\begingroup$ Nice answer. (Would be better with links to the data, but interesting nonetheless.) At a high level, I'm sensing processing speed and throughput are only part of the problem, and this aspect relates more to time than anything else (i.e. how long to process for a given decision.) But I don't believe the recent breakthroughs in NNs are predominantly due to fast processors. Rather, these breakthroughs seem related to new techniques and increasing algorithmic sophistication. My sense is the same holds for organic brains. Machines, but incredibly sophisticated and optimized $\endgroup$ – DukeZhou Apr 3 '18 at 19:55
  • $\begingroup$ @DukeZhou Links added. No, you are quite mistaken. The claim that the brain is just a Turing Machine can be disproved. Also, I wouldn't use the term 'breakthrough' for a system that can be beaten by a six month old baby. $\endgroup$ – scientious Apr 4 '18 at 16:54
1
$\begingroup$

Short answer: nobody knows. Long answer: all strong-AI works. However, to write something useful to the o.p., say that the question contains several implicit statements, analyze them could be useful to clarify the issue:

a) why thing that 1 transistor has the same functionality than 1 neuron ? Some obvious differences: a transistor has 3 legs, each neuron has around 7000 synapses; a transistor has 3 layers of material, a neuron is a full micro-machine with thousands of components; each synapses itself is a switch, connected to one or more other cells, and can produce different kinds of signals (activation/inhibitory, frequencies, amplitude, ...).

b) compare memory amounts: the amount of memory in a person equivalent to the one in a computer s 0 bytes, we are not able to remember any thing forever and without distortion. Human memory is symbolic, temporal, associative, influenced by body and feelings, ... . Something totally different than computers one.

c) all previous are about "hardware": if we analyze software and training, differences are even bigger. Even assume than intelligence is placed only in the brain, forgetting the role of hormonal system, senses, ... is a simplification not yet proof.

In conclusion: human mind is totally different from a computer, we are far to understand it, and more far to replicate it.

From the start of computers age, the idea than intelligence will appear when the amounts of memory, process power, ... reaches some threshold has became false.

$\endgroup$
  • $\begingroup$ You didn't get my question, I asked about brain's power not how the brain functions (i know we dont understand it clearly). Your 1st point, normal transistor is much older nowadays we use FET's which is smaller. A human neuron is a few centimeters long, millions of fet's can be fitted in those dimensions easily replicating a neuron. 2nd point it is advantageous to have distortion less memory and in much greater amt which an animal doesn't possess. And third I am only asking the power that is i9/i7/in processor not how you are using the processor..But overall i get what you are trying to say $\endgroup$ – DuttaA Feb 9 '18 at 20:19
  • $\begingroup$ Still same mistake: you want to compare "brain power", undefined term. The "brain power" of a computer is 0, the "computer power" of brain is also 0 $\endgroup$ – pasaba por aqui Feb 9 '18 at 20:21
  • $\begingroup$ I actually am not comparing brain with computer i am asking where is the problem in recreating a brain $\endgroup$ – DuttaA Feb 9 '18 at 20:24
  • 1
    $\begingroup$ Recreating each brain atom, each brain protein, each gen, each synapse, each neuron ? Modeling them as ? I will give you an example: C. Elegans is an small worm with only 300 neurons and 7000 synapses, already totally mapped and listes, but nobody as yet reproduced nor simulated it. $\endgroup$ – pasaba por aqui Feb 9 '18 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.