9
$\begingroup$

Based on fitting to historical data and extrapolation, when is it expected that the number of neurons in AI systems will equal those of the human brain?

I'm interested in a possible direct replication of the human brain, which will need equal numbers of neurons.

Of course, this assumes neurons which are equally capable as their biological counterparts, which development may happen at a faster or slower rate than the quantitative increase.

Improve this question
$\endgroup$
1
  • $\begingroup$ Can you add information about why you think this is importan-- toward what end does the answer serve? I do see some useful information in answers below, but to date I think the best response is @Ankur 's, which points out the answer is without utility unless we define what it means to have a certain number of neurons. $\endgroup$
    – dynrepsys
    Nov 14, 2016 at 14:27

6 Answers 6

Reset to default
17
$\begingroup$

Some back of the envelope calculations :

number of neurons in AI systems

The number of neurons in AI systems is a little tricky to calculate, Neural Networks and Deep Learning are 2 current AI systems as you call them, specifics are hard to come by (If someone has them please share), but data on parameters do exist, parameters are more analogous to synapses (connections) than neurons (the nodes in between connections) somewhere in the range of 100-160 billion is the current upper number for specialized networks.

Deriving the number of neurons in AI systems from this number is a stretch since these AIs emulate certain types of connections and sub assemblies of neurons, but let's continue...

equal those of the human brain?

So now let's look at the brain, and again this are all contested numbers. Number of neurons ~ 86 Billion, Number of Synapses ~ 150 Trillion, another generalization: average number of synapses per neuron ~ 1,744.

So now we have something to compare, and I can't stress this enough, these are all wonky numbers, so let's make our life a little easier and divide :

Number of Synapses (Brain ) : 150 trillion / Number of parameters AIs : 150 billion = 1,000 or in other words current AIs would have to scale by a factor of one thousand their connections to be on par with the brain...

Number of Neurons (Brain ) : 86 Billion / Number of Neurons AIs ( 150 billion / 1,744 ) = 86 Million equivalent AI Neurons

Which makes sense, mathematically at least : you can multiply the factor ( 1000 ) times the current number of equivalent AI Neurons ( 86 million) to get the number of neurons in the human brain (86 Billion)

When ?

Well,let's use moore's law ( number of transistors processing power doubles about every 2 years ) as a rough measure of technological progress: 

     #AI NEURONS            YEAR
     86,000,000             2016
     172,000,000            2018
     344,000,000            2020
     688,000,000            2022
     1,376,000,000          2024
     2,752,000,000          2026
     5,504,000,000          2028
     11,008,000,000         2030
     22,016,000,000         2032
     44,032,000,000         2034
     88,064,000,000         2036


     # NEURONS HUMAN BRAIN 
     86,000,000,000

So, if all this made sense to you, somewhere around the year 2035.

Improve this answer
$\endgroup$
3
  • $\begingroup$ arxiv.org/abs/1701.06538 , this might incline you revise your estimate. ;-) They want to scale up to a trillion parameters and I wouldn't be surprised if they did it this year, which would be a jump to 2022 on your timeline. $\endgroup$
    – BlindKungFuMaster
    Feb 22, 2017 at 12:18
  • $\begingroup$ is the number of parameters in this analogy equivalent to the number of synapses? $\endgroup$
    – committedandroider
    Sep 25, 2019 at 19:16
  • $\begingroup$ @committedandroider: roughly but yes, parameters here are weights in between neurons/nodes in ANN, in natural NN the equivalent is synaptic weights, which map to synapses. $\endgroup$
    – Keno
    Sep 26, 2019 at 20:18
7
$\begingroup$

Soon enough but that doesn't mean anything at all. In machine learning the word neuron represents a calculation whereas in brain the word neuron represent a specific type of cell which is a biochemical system.

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ Yes, plus multiple calculations are carried out within biological neurons through the interactions of a dizzying array of neurotransmitters etc. It might take multiple hardware or software neurons to get the same effect in an artificial neural net. Furthermore, we don't know how many of these neurochemical exchanges of information are taking place - new ones are being discovered all the time - and have a poor understanding of what their functions are. So we can't even give an accurate number of how many processing units we need, let alone estimate when we'll have them. $\endgroup$
    – SQLServerSteve
    Nov 14, 2016 at 6:50
  • $\begingroup$ A biochemical system that performs performs a complex calculation with a chemical return value... $\endgroup$
    – Tom Hale
    Nov 14, 2016 at 7:39
  • $\begingroup$ @TomHale What is your reasoning that made you reach the conclusion that the biochemical system is performing a calculation? Also, given any system how would you decide if it is performing some calculation or not? $\endgroup$
    – Ankur
    Nov 14, 2016 at 11:46
  • $\begingroup$ It's performing a transformation of inputs based on its current biological state and the "rules" of the biological processes, and produces outputs $\endgroup$
    – Tom Hale
    Nov 14, 2016 at 12:45
  • $\begingroup$ @TomHale: Ok, now consider this scenario - I have a bucket with bunch of holes at the bottom, now if I start pouring water in the bucket, the water will start to come out of the holes at the bottom - There is input and there is output. Would you consider the bucket doing computation? $\endgroup$
    – Ankur
    Nov 14, 2016 at 14:38
3
$\begingroup$

The answers so far haven't answered the question numerically, so here is my attempt to steer them in the direction I was seeking:

The freely available Deep Learning Book has the following figure on page 27:

Size of neural nets over time

I question the blue fit line, as it seems that data points may be better described by a parabolic or exponential function.

In any case, based upon this conservative linear fit, the authors predict that the number of neurons in a ANN will equal that of the human brain in 2056.

The referenced nerual networks are:

Neural networks referred to in image above

What is interesting to note that when The Singularity is Near was written in 2006, Ray Kurzweil said that the refractory period of a biological neuron was already 1,000,000 times slower than that of an artificial one.

Improve this answer
$\endgroup$
4
  • $\begingroup$ @Keno is referring to this answer $\endgroup$
    – Tom Hale
    Nov 14, 2016 at 12:47
  • 1
    $\begingroup$ Revised my table numbers down, so year 2035 is my current estimate. $\endgroup$
    – Keno
    Nov 22, 2016 at 19:46
  • $\begingroup$ @TomHale - regarding - "Ray Kurzweil said that the refractory period of a biological neuron was already 1,000,000 times slower than that of an artificial one. " - does that mean the artificial neurons will be able to do more computations? $\endgroup$
    – committedandroider
    Sep 25, 2019 at 19:12
  • 1
    $\begingroup$ @committedandroider if the complexity of an artificial neuron could be increased to that of a biological one, while keeping the same speed multiplier, then I don't see why not. Also, artificial neurons could be electrical whereas there is a slower chemical component to brain neurons, AFAIK. $\endgroup$
    – Tom Hale
    Sep 26, 2019 at 8:06
3
$\begingroup$

While interesting, this is all rendered somewhat moot if you think about what will happen once we understand how the brain works. After all, once we understood flight, we didn't start making birds. The same goes for AI. Here are just a few ways in which human brains and digital brains can't be compared.

The digital brain won't have to worry about food and drink. They will also be more reliable (or less redundant) as electronics is way more reliable than neurons (a guess). Digital brains will also be able to share learning and information. Once one Model 3X digital brain has learned something, the others need merely to have the bits uploaded. Sure, it will be more complicated than that but, remember, we will know how it all works so merging the experiences of one digital being with another should be doable. If we want our digital brain to have symbolic algebra ability, we will have to teach it some things but we can also hard-wire it to Mathematica or the like.

In short, it will be like apples and oranges.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Nice answer. Welcome to AI! There is a lot of interest in translating/migration human minds to computing mediums in a hypothetical post-singularity context. Wondered if you had any thoughts on the idea of maintaining a software kernel based on human brain for whatever reason, even if that model becomes technically obsolete in terms of raw intelligence. $\endgroup$
    – DukeZhou
    Aug 29, 2017 at 0:44
  • 1
    $\begingroup$ electronics is way more reliable than neurons - I'd love to see a complex electronic system that can function nonstop for nearly a century with absolutely zero downtime (general anesthesia notwithstanding). $\endgroup$
    – forest
    Jul 9, 2019 at 10:00
  • $\begingroup$ I was thinking about this too - "The digital brain won't have to worry about food and drink." . If we don't have to model every human behavior, I'am interested in how neurons we would need to model the human brain $\endgroup$
    – committedandroider
    Sep 25, 2019 at 19:14
1
$\begingroup$

The human brain contains billions of neurons, which means we won't be making one tomorrow. However, technology tends to advance in an exponential manner, and that may soon be a real possibility. Also, the idea of making an artificial human brain would not only take more neurons than a current average computer could process, or we could make outside of computers, but we also need an understanding of the human brain. There is only one animal with neurons that we have completed a full connectome of and that is the Caenorhabditis elegans (roundworm) and it has less than 500 neurons. It may be a while before we actually make a human brain, but within 30 years is a reasonable estimation with the rate that technology improves now.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Welcome to AI.SE :) Please note that you didn't actually answer the question... please re-read the first 10 words. $\endgroup$
    – Tom Hale
    Nov 28, 2016 at 4:17
  • $\begingroup$ @TomHale oops thanks I didn't realize. I'll fix that when I get a chance. Thanks for pointing it out. :) $\endgroup$
    – I. Golsby
    Nov 29, 2016 at 1:44
  • $\begingroup$ Is there a list that maps out how many neurons each animal has. I'am thinking we could work our own way up from the roundworm to the fruit fly then to a rat... $\endgroup$
    – committedandroider
    Sep 25, 2019 at 19:07
0
$\begingroup$

2035, 2056? Those predictions are hilarious :)

2019 - 1,6Billion parameter model (GPT-2)

2020 -175Billion parameter model (GPT-3) more than 100x jump in a year

2021(April) - "Microsoft's ZeRO-Infinity can now run a model with over a trillion parameters on a single NVIDIA DGX-2 node and over 30 trillion parameters on 32 nodes (512 GPUs). With a hundred DGX-2 nodes in a cluster, Microsoft projects ZeRO-Infinity can train models with over a hundred trillion parameters"

https://www.microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-training/

So in 2021 we now have tech to train 30Trillion- 100 Trillion parameters/neurons Model

100 trillion = Human Brain

With this tech, OpenAI together with Microsoft, other company or gov will train 100Trillion model in 2021 or 2022 at the latest

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Can you provide a reference that supports this claim "100 trillion = Human Brain"? $\endgroup$
    – nbro
    Apr 23, 2021 at 14:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .