# Tag Info

10

H+ magazine wrote an estimate in 2009 that seems broadly comparable to other things I've seen; they think the human brain is approximately 37 petaflops. A supercomputer larger than that 37 petaflop estimate exists today. But emulation is hard. See this SO question about hardware emulation or this article on emulating the SNES, in which they require 140 ...

6

In May 2016 Google announced a custom ASIC which was is specifically built for machine learningwiki and tailored for TensorFlow. It is using tensor processing unit (TPU) which is a programmable microprocessor designed to accelerate artificial neural networks. NeuroCores, 12x14 sq-mm chips which can be interconnected in a binary tree, see: Neurogrid, a ...

6

The human brain contains about 100 billion neurons ($10^{11}$) and about a hundred trillion synapses ($10^{14}$). Each neuron can fire about 100 times a second. If we model the brain as a simple neural network, then it would be equivalent to a machine that requires 1016 calculations per second and 1013 bits of memory. From Wikipedia Kurzweil introduces the ...

6

The approach you describe is called neuromorphic computing and it's quite a busy field. IBM's TrueNorth even has spiking neurons. The main problem with these projects is that nobody quite knows what to do with them yet. These projects don't try to create chips that are optimised to run a neural network. That would certainly be possible, but the ...

5

XPU is a device abstraction for Intel heterogeneous computation architectures, which can be mapped to CPU, GPU, FPGA and other accelerators. The "X" from XPU is just like a variable, like in maths, so you can do X=C and you get CPU accceleration, or X=G and you get GPU acceleration... That's the intuition behind that abstract name. In order to ...

5

Tensor operations The major work in most ML applications is simply a set of (very large) tensor operations e.g. matrix multiplication. You can do that easily in an ASIC, and all the other algorithms can just run on top of that.

4

If neurons and synapses can be implemented using transistors, I hope you are not talking about the neural networks which are currently winning all competitions in machine learning (MLPs, CNNs, RNNs, Deep Residual Networks, ...). Those were once used as a model for neurons, but they are only very loosely related to what happens in real brain cells. Spiking ...

3

Google has not released the manufacturing details for their TPUs. However, it's suspected that they're produced by either Taiwan Semiconductor Manufacturing or GlobalFoundries, as these are some of the largest companies in the industry. Source

3

I did some recent research on this topic. It all comes down to parallelization. Basically there are 2 ways to do it: model parallelization or batch parallelization. Model parallelization is when you split the model by layers among multiple GPUs. As per my best knowledge you can't split a layer between GPUs, so 8 GPUs would serve 8 layers that is very ...

3

I think the algorithm has changed minimally, but the necessary hardware has been trimmed to the bone. The number of gate transitions are reduced (perhaps float ops and precision too), as are the number of data move operations, thus saving both power and runtime. Google suggests their TPU achieves a 10X cost saving to get the same work done. https://...

2

Specialized AI hardware takes advantage of highly parallelizable nature of many neural network designs. GPUs are designed for pixel crunching - coincidentally very paralelisable too. This is why they often offer orders of magnitude better performance (in NN tasks) than CPUs. That being said, GPUs have shortcomings to. First, many of GPU's built in features ...

2

Parallel processing is very important when computing ANNs. Though they are not designed for AIs, GPUs are widely used in machine learning because of their higher capability of paralel processing and it is due to their higher number of cores comparing CPUs. All in all, the dedicated hardware should have GPU-like architecture.

2

From the Deep Stack paper: This seems to be for training: For the turn network, ten million poker turn situations (from after the turn card is dealt) were generated and solved with 6,144 CPU cores of the Calcul Quebec MP2 research cluster, using over 175 core years of computation time. For the flop network, one million poker flop situations (from after ...

2

Theoretically, there shouldn't be a problem copying either of the artificial brains in any state. Difficulty in measuring a state doesn't seem to really be a problem until you get down to the quantum level, where the means of measurement affect the state. The configuration of the artificial brains, including pathway structures and states, should be ...

2

While a single transistor could approximate the basic function of a single neuron, I cannot agree that any electronic element could simulate the synapses/axons. Transistors are etched on a flat surface, and could be interconnected only to adjacent or close by transistors. Axons in the brain span huge distances (compared to the size of the neuron itself), and ...

2

Not just how much, but what kind of processing power : there're specially-crafted dedicated chips, and it has a practical applications, so it's not a lab-only project

2

In the perceptron design generally used in Artificial Neural Networks, we know precisely what a single neuron is capable of computing. It can compute a function $$f(x) = g(w^{\top} x),$$ where $x$ is a vector of inputs (may also be vector of activation levels in previous layer), $w$ is a vector of learned parameters, and $g$ is an activation function. We ...

2

Neural networks are just software. Software is just one form of data in the Von Neumann Architecture. Most of the data is stored on the disk. There are two common types of disks: HDD: An actual disk which can be magnetified is spinning. SSD: Solid-state drive If the part where the data is stored is destroyed, the data is destroyed. Of course, the data can ...

2

Training time depends on a lot of parameters. Some of them are: Size of each image (resolution) Color/Monochrome image (color image has 3 times data if you consider RGB image) Like you mentioned on the type of DNN. No. of layers of DNN. No. of neurons in each layer. Total no. of images in the dataset. (2.6 million here) GPU you are using (you didn't ...

2

Computational Creativity is not an unassailable challenge (depending on who you talk to;) Philosophers have claimed algorithms can't be creative, but Marcel Duchamp, one of the most significant artists in modernity, famously stated that: "All artists are not chess players, but all chess players are artists" This would seem to have been validated by ...

1

YELP Dataset (200k images) used to take 5 hr for training to identify Five (5) classes on GPU - Nvidia 1080 Ti with 11 GB RAM. So I guess in your case it will take days. Again it will depend on the type of your GPU configuration and type of Architecture you will be using.

1

Deep models are very tolerant to arithmetic underflow. You can hope for neglectable differences in prediction accuracy between FP32 and FP16 models. Check this paper for concrete results.

1

You can expect that the inference time will strongly depend on particular hardware and software present on your platform. First, GPU equipped devices (eg NVidia TX) will outperform non-GPU equipped devices (eg. Intel Movidius). Second, software support (eg. cudnn, TensorRT) will make dramatic further impact. For instance, we have measured the inference ...

1

Then work with quantized network it's good idea to normalize block of them them with floating point scaling factor (16 bit float for example). The weight channel and input channel will have represenation S_f * w_n where S_f - scalar float and w_n - low-bit fixed or integer tensor or vector. Then calculating dot product put it into high-bit fixed/integer ...

1

There are a lot of other potential applications. It's a good idea to start with GPU related problems, since GPUs are essentially doing a slightly wider set of operations, slightly slower. Some possible problems where TPUs might be advantageous are: Shaders are algorithms for rendering graphics in one style or another. Since computer graphics can be ...

1

In my opinion, there are many functions in our brain. Surely much more than the artificial neural network nowadays. I guess this is the field of brain science or cognitive psychology. Some brain structures may help for certain applications, but not all. Neural network though is a simplest form of our brain, but has the most general usages. On the other ...

1

The building unit of a neural network is called perceptron. It cannot be represented by single transistor because it should hold arbitrary (float) value, over multiple computational iterations. (While the transistor is only binary, and does not work as memory on its own.) Furthermore, the strengths of the NN is in it's flexibility, which you would lose if ...

Only top voted, non community-wiki answers of a minimum length are eligible