8
$\begingroup$

How big artificial neural networks can we run now (either with full train-backprop cycle or just evaluating network outputs) if our total energy budget for computation is equivalent to the human brain energy budget (12.6 watts)?

Let assume one cycle per second, which seems to roughly match the firing rate of biological neurons.

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
8
$\begingroup$

126 million artificial neurons at 12.6 Watts, with IBM's True North

Back in 2014, IBM's True North chip was pushing 1 million neurons at less than 100mW.

So that's roughly 126 million artificial neurons at 12.6 Watts.

A mouse has 70 million neurons.

IBM believes they can build a human-brain scale True North mainframe at a "mere" 4kW.

Once 3D transistors come to market, I think we'll catch up to animal brain efficiency pretty fast.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ > So that's roughly 126 million artificial neurons at 12.6 Watts. > A mouse has 70 million neurons. Problem is, we don't know if a single artificial "neuron" is truly a one to one equivalent for a real biological neuron. $\endgroup$
    – mindcrime ♦
    Commented Sep 2, 2016 at 22:52
  • $\begingroup$ In addition to mindcrime, keep in mind that for roughly 60 years, neuroscientists/AI researchers have consistently badly underestimated the processing power of the human brain. I saw early estimates by big names that our minds could only store a few 100 MB! We have yet to factor in things like bit depth, proper architecture, possible quantum states in brain cells, etc. IBM may likely build a power-efficient 70-million-neuron neural net that can't even approximate the behavior of a mouse's parasites. AI researchers have historically always badly overestimated their own capabilities. $\endgroup$
    – SQLServerSteve
    Commented Sep 2, 2016 at 23:24
  • $\begingroup$ Indeed, forgot about IBM chips. I didn't even take them into consideration when writing the question, as there is known criticism on their efficiency (see e.g. facebook.com/yann.lecun/posts/10152184295832143). Though technically they still implement neural networks, so +1 from me. $\endgroup$
    – liori
    Commented Sep 2, 2016 at 23:51
  • $\begingroup$ @mindcrime & SQLServerSteve Well, the question could have been specifically about vanilla feed forward ANNs running on stock hardware. That's not the impression I got from the question though. $\endgroup$
    – Doxosophoi
    Commented Sep 3, 2016 at 0:37
  • $\begingroup$ Nor did it seem to be about artificial neural networks that have yet to be imagined. $\endgroup$
    – Doxosophoi
    Commented Sep 3, 2016 at 0:48
3
$\begingroup$

If you limited yourself to 12.6 watts, you wouldn't get much done. Just lookup the power consumption for a modern GPU, look at the size networks people are training on those, and then scale down. For reference, modern GPU's appear to consume between 52-309 watts under heavy use.

Clearly energy efficiency is one area where the human brain is still far head of ANN's.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .