Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more
11 votes
Accepted

How powerful a computer is required to simulate the human brain?

H+ magazine wrote an estimate in 2009 that seems broadly comparable to other things I've seen; they think the human brain is approximately 37 petaflops. A supercomputer larger than that 37 petaflop ...
Matthew Gray's user avatar
  • 4,262
7 votes
Accepted

Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?

The approach you describe is called neuromorphic computing and it's quite a busy field. IBM's TrueNorth even has spiking neurons. The main problem with these projects is that nobody quite knows ...
BlindKungFuMaster's user avatar
6 votes
Accepted

What exactly is an XPU?

XPU is a device abstraction for Intel heterogeneous computation architectures, which can be mapped to CPU, GPU, FPGA and other accelerators. The "X" from XPU is just like a variable, like in ...
JVGD's user avatar
  • 1,138
6 votes
Accepted

Are there any microchips specifically designed to run ANNs?

In May 2016 Google announced a custom ASIC which was is specifically built for machine learningwiki and tailored for TensorFlow. It is using tensor processing unit (TPU) which is a programmable ...
kenorb's user avatar
  • 10.5k
6 votes

How powerful a computer is required to simulate the human brain?

The human brain contains about 100 billion neurons ($10^{11}$) and about a hundred trillion synapses ($10^{14}$). Each neuron can fire about 100 times a second. If we model the brain as a simple ...
Vishnu JK's user avatar
  • 1,082
5 votes
Accepted

How does using ASIC for the acceleration of AI work?

Tensor operations The major work in most ML applications is simply a set of (very large) tensor operations e.g. matrix multiplication. You can do that easily in an ASIC, and all the other algorithms ...
Peteris's user avatar
  • 883
4 votes

Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?

If neurons and synapses can be implemented using transistors, I hope you are not talking about the neural networks which are currently winning all competitions in machine learning (MLPs, CNNs, RNNs, ...
Martin Thoma's user avatar
  • 1,055
3 votes

Who manufactures Google's Tensor Processing Units?

Google has not released the manufacturing details for their TPUs. However, it's suspected that they're produced by either Taiwan Semiconductor Manufacturing or GlobalFoundries, as these are some of ...
Peyton B's user avatar
3 votes

Are more than 8 high performance Nvidia GPUs practical for deep learning applications?

I did some recent research on this topic. It all comes down to parallelization. Basically there are 2 ways to do it: model parallelization or batch parallelization. Model parallelization is when ...
Manngo's user avatar
  • 296
3 votes

How powerful is the machine that beat the poker professional players recently?

From the Deep Stack paper: This seems to be for training: For the turn network, ten million poker turn situations (from after the turn card is dealt) were generated and solved with 6,144 CPU cores ...
BlindKungFuMaster's user avatar
3 votes

How does using ASIC for the acceleration of AI work?

I think the algorithm has changed minimally, but the necessary hardware has been trimmed to the bone. The number of gate transitions are reduced (perhaps float ops and precision too), as are the ...
Randy's user avatar
  • 679
2 votes

What is the difference between a normal processor and a processor designed for AI?

Specialized AI hardware takes advantage of highly parallelizable nature of many neural network designs. GPUs are designed for pixel crunching - coincidentally very paralelisable too. This is why they ...
Andrew Butenko's user avatar
2 votes

What is the difference between a normal processor and a processor designed for AI?

Parallel processing is very important when computing ANNs. Though they are not designed for AIs, GPUs are widely used in machine learning because of their higher capability of paralel processing and ...
CaptainVice's user avatar
2 votes
Accepted

If we achieve sentience using mutable hardware, will it be possible to make a copy of that "brain" and its active state?

Theoretically, there shouldn't be a problem copying either of the artificial brains in any state. Difficulty in measuring a state doesn't seem to really be a problem until you get down to the ...
DukeZhou's user avatar
  • 6,237
2 votes

Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?

While a single transistor could approximate the basic function of a single neuron, I cannot agree that any electronic element could simulate the synapses/axons. Transistors are etched on a flat ...
Iliyan Bobev's user avatar
2 votes

How powerful a computer is required to simulate the human brain?

Not just how much, but what kind of processing power : there're specially-crafted dedicated chips, and it has a practical applications, so it's not a lab-only project
Alexey Vesnin's user avatar
2 votes

Are artificial networks based on the perceptron design inherently limiting?

In the perceptron design generally used in Artificial Neural Networks, we know precisely what a single neuron is capable of computing. It can compute a function $$f(x) = g(w^{\top} x),$$ where $x$ ...
Dennis Soemers's user avatar
  • 10.3k
2 votes
Accepted

Does software remain even when hardware is demolished?

Neural networks are just software. Software is just one form of data in the Von Neumann Architecture. Most of the data is stored on the disk. There are two common types of disks: HDD: An actual disk ...
Martin Thoma's user avatar
  • 1,055
2 votes
Accepted

How long it takes to train face recognition deep neural network? (rough estimation)

Training time depends on a lot of parameters. Some of them are: Size of each image (resolution) Color/Monochrome image (color image has 3 times data if you consider RGB image) Like you mentioned on ...
Kartik Podugu's user avatar
2 votes
Accepted

How do neural network topologies affect GPU/TPU acceleration?

The topology of a neural network can have a significant impact on the performance of GPU and TPU acceleration. The most ...
Faizy's user avatar
  • 1,114
2 votes

For an LLM model, how can I estimate its memory requirements based on storage usage?

It varies depending on various factors such as quantization. My rough rule of thumb is memory need is 2-4x of the disk size. Just as an example, the model at https://huggingface.co/TheBloke/wizardLM-...
beejay's user avatar
  • 21
2 votes

For an LLM model, how can I estimate its memory requirements based on storage usage?

It really depends on what you want to do with it, how you load it and what else you add. For inference at half-precision (16 bit per param) a 7B param models should be 7 billion * 2 bytes per param or ...
Andrew Mellinger's user avatar
1 vote
Accepted

What kind of NN I need to find ideal ranges and correlation between them?

What you described seems like a pretty standard binary classification problem. There are many good algorithms, that are much simpler and more interpretable than NNs. I don't see why you would straight ...
ImotVoksim's user avatar
1 vote

How long it takes to train face recognition deep neural network? (rough estimation)

YELP Dataset (200k images) used to take 5 hr for training to identify Five (5) classes on GPU - Nvidia 1080 Ti with 11 GB RAM. So I guess in your case it will take days. Again it will depend on the ...
Sanmati Jain's user avatar
1 vote

Does fp32 & fp64 performance of GPU affect deep learning model training?

Deep models are very tolerant to arithmetic underflow. You can hope for neglectable differences in prediction accuracy between FP32 and FP16 models. Check this paper for concrete results.
ssegvic's user avatar
  • 499
1 vote

What are the aspects that most impact on the inference time for neural networks in embedded systems?

You can expect that the inference time will strongly depend on particular hardware and software present on your platform. First, GPU equipped devices (eg NVidia TX) will outperform non-GPU equipped ...
ssegvic's user avatar
  • 499
1 vote

Tips for keeping the distribution of weights normal

Then work with quantized network it's good idea to normalize block of them them with floating point scaling factor (16 bit float for example). The weight channel and input channel will have ...
mirror2image's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible