11
votes
Accepted
How powerful a computer is required to simulate the human brain?
H+ magazine wrote an estimate in 2009 that seems broadly comparable to other things I've seen; they think the human brain is approximately 37 petaflops. A supercomputer larger than that 37 petaflop ...
- 4,262
6
votes
Accepted
What exactly is an XPU?
XPU is a device abstraction for Intel heterogeneous computation architectures, which can be mapped to CPU, GPU, FPGA and other accelerators. The "X" from XPU is just like a variable, like in ...
- 1,098
6
votes
Accepted
Are there any microchips specifically designed to run ANNs?
In May 2016 Google announced a custom ASIC which was is specifically built for machine learningwiki and tailored for TensorFlow. It is using tensor processing unit (TPU) which is a programmable ...
- 10.4k
6
votes
How powerful a computer is required to simulate the human brain?
The human brain contains about 100 billion neurons ($10^{11}$) and about a hundred trillion synapses ($10^{14}$). Each neuron can fire about 100 times a second. If we model the brain as a simple ...
- 1,062
6
votes
Accepted
Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?
The approach you describe is called neuromorphic computing and it's quite a busy field.
IBM's TrueNorth even has spiking neurons.
The main problem with these projects is that nobody quite knows ...
- 4,205
5
votes
Accepted
How does using ASIC for the acceleration of AI work?
Tensor operations
The major work in most ML applications is simply a set of (very large) tensor operations e.g. matrix multiplication. You can do that easily in an ASIC, and all the other algorithms ...
- 883
4
votes
Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?
If neurons and synapses can be implemented using transistors,
I hope you are not talking about the neural networks which are currently winning all competitions in machine learning (MLPs, CNNs, RNNs, ...
- 1,055
3
votes
Who manufactures Google's Tensor Processing Units?
Google has not released the manufacturing details for their TPUs. However, it's suspected that they're produced by either Taiwan Semiconductor Manufacturing or GlobalFoundries, as these are some of ...
- 81
3
votes
Are more than 8 high performance Nvidia GPUs practical for deep learning applications?
I did some recent research on this topic. It all comes down to parallelization.
Basically there are 2 ways to do it: model parallelization or batch parallelization.
Model parallelization is when ...
- 296
3
votes
How powerful is the machine that beat the poker professional players recently?
From the Deep Stack paper:
This seems to be for training:
For the turn network, ten million poker turn situations (from after the turn card is dealt) were generated and solved with 6,144 CPU cores ...
- 4,205
3
votes
How does using ASIC for the acceleration of AI work?
I think the algorithm has changed minimally, but the necessary hardware has been trimmed to the bone.
The number of gate transitions are reduced (perhaps float ops and precision too), as are the ...
- 679
2
votes
What is the difference between a normal processor and a processor designed for AI?
Specialized AI hardware takes advantage of highly parallelizable nature of many neural network designs. GPUs are designed for pixel crunching - coincidentally very paralelisable too. This is why they ...
- 221
2
votes
What is the difference between a normal processor and a processor designed for AI?
Parallel processing is very important when computing ANNs. Though they are not designed for AIs, GPUs are widely used in machine learning because of their higher capability of paralel processing and ...
- 21
2
votes
Accepted
If we achieve sentience using mutable hardware, will it be possible to make a copy of that "brain" and its active state?
Theoretically, there shouldn't be a problem copying either of the artificial brains in any state.
Difficulty in measuring a state doesn't seem to really be a problem until you get down to the ...
- 6,237
2
votes
Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?
While a single transistor could approximate the basic function of a single neuron, I cannot agree that any electronic element could simulate the synapses/axons. Transistors are etched on a flat ...
- 276
2
votes
How powerful a computer is required to simulate the human brain?
Not just how much, but what kind of processing power : there're specially-crafted dedicated chips, and it has a practical applications, so it's not a lab-only project
- 197
2
votes
Are artificial networks based on the perceptron design inherently limiting?
In the perceptron design generally used in Artificial Neural Networks, we know precisely what a single neuron is capable of computing. It can compute a function
$$f(x) = g(w^{\top} x),$$
where $x$ ...
- 10.1k
2
votes
Accepted
Does software remain even when hardware is demolished?
Neural networks are just software. Software is just one form of data in the Von Neumann Architecture.
Most of the data is stored on the disk. There are two common types of disks:
HDD: An actual disk ...
- 1,055
2
votes
Accepted
How long it takes to train face recognition deep neural network? (rough estimation)
Training time depends on a lot of parameters. Some of them are:
Size of each image (resolution)
Color/Monochrome image (color image has 3 times data if you consider RGB image)
Like you mentioned on ...
- 174
2
votes
Accepted
How do neural network topologies affect GPU/TPU acceleration?
The topology of a neural network can have a significant impact on the performance of GPU and TPU acceleration.
The most ...
- 1,094
2
votes
For an LLM model, how can I estimate its memory requirements based on storage usage?
It varies depending on various factors such as quantization. My rough rule of thumb is memory need is 2-4x of the disk size. Just as an example, the model at https://huggingface.co/TheBloke/wizardLM-...
- 21
1
vote
Accepted
What kind of NN I need to find ideal ranges and correlation between them?
What you described seems like a pretty standard binary classification problem. There are many good algorithms, that are much simpler and more interpretable than NNs. I don't see why you would straight ...
- 76
1
vote
How long it takes to train face recognition deep neural network? (rough estimation)
YELP Dataset (200k images) used to take 5 hr for training to identify Five (5) classes on GPU - Nvidia 1080 Ti with 11 GB RAM. So I guess in your case it will take days. Again it will depend on the ...
- 11
1
vote
Does fp32 & fp64 performance of GPU affect deep learning model training?
Deep models are very tolerant to arithmetic underflow. You can hope for neglectable differences in prediction accuracy between FP32 and FP16 models. Check this paper for concrete results.
- 499
1
vote
What are the aspects that most impact on the inference time for neural networks in embedded systems?
You can expect that the inference time will strongly depend on particular hardware and software present on your platform. First, GPU equipped devices (eg NVidia TX) will outperform non-GPU equipped ...
- 499
1
vote
Tips for keeping the distribution of weights normal
Then work with quantized network it's good idea to normalize block of them them with floating point scaling factor (16 bit float for example).
The weight channel and input channel will have ...
- 705
1
vote
Which artificial intelligence algorithms could use tensor specific hardware?
There are a lot of other potential applications. It's a good idea to start with GPU related problems, since GPUs are essentially doing a slightly wider set of operations, slightly slower. Some ...
- 9,207
Only top scored, non community-wiki answers of a minimum length are eligible