It just struck me that the human brain has about as many neurons as an Nvidia M100 chip has transistors, around 8E10. Obviously, a transistor is not equivalent to a neuron; but Elon Musk's xAI is planning to build a cluster with 100,000 such processors.

I'm not familiar with the hardware topology of such clusters; but would it be fair to say that this cluster has a "complexity" (however vaguely defined) in the order of magnitude of the human brain?

There is a caveat here in that comparing computational hardware with biological wetware, like in the above question, is comparing apples to pears: The brain's physical basis is a neural net in the truest sense of the word while the computational hardware of xAI's cluster will run an emulation of one, even though it is dedicated hardware supporting this specific purpose.

I hope it's OK to ask this related question in the same post: Is the complexity of the model such a cluster might run/train comparable to the brain's neural network?

  • $\begingroup$ you should also consider the connectivity, and then realize there is no such thing in computers, if you don't specify a model that exploits the computation $\endgroup$
    – Alberto
    Commented Jul 10 at 20:13
  • $\begingroup$ @Alberto Hm... there clearly is connectivity in computers (i.e., in their hardware). $\endgroup$ Commented Jul 10 at 20:57
  • $\begingroup$ what do you mean? you don't need electricity to make computers, you can make turing machines out of water and tubes, but you would never say that the world water infrastructure has "the complexity of a brain" $\endgroup$
    – Alberto
    Commented Jul 10 at 21:41
  • $\begingroup$ @Alberto The national water infrastructure of an industrialized country (here is an image of the water backbone network for a German state) may be of comparable complexity as the connectome of a nematode, I think. Electricity networks, being meshed more, will have a greater complexity. The topology likely differs dramatically, sure (most distribution networks will be more hierarchical than a connectome, I suppose), but complex they are. $\endgroup$ Commented Jul 11 at 7:15


You must log in to answer this question.