# How does using ASIC for the acceleration of AI work?

We can read on Wikipedia page that Google built a custom ASIC chip for machine learning and tailored for TensorFlow which helps to accelerate AI.

Since ASIC chips are specially customized for one particular use without the ability to change its circuit, there must be some fixed algorithm which is invoked.

So how exactly does the acceleration of AI using ASIC chips work if its algorithm cannot be changed? Which part of it is exactly accelerating?

• No mainstream AI technique that I'm aware of requires modification of the algorithm, though most are dependent on the ability to modify data (connection strengths, population members) etc. – NietzscheanAI Aug 17 '16 at 12:00
• So the only dynamic part like state of network is kept on some flash memory or drive? – kenorb Aug 17 '16 at 12:13
• According to en.wikipedia.org/wiki/Application-specific_integrated_circuit, modern ASICs can have RAM... – NietzscheanAI Aug 17 '16 at 12:16

## Tensor operations

The major work in most ML applications is simply a set of (very large) tensor operations e.g. matrix multiplication. You can do that easily in an ASIC, and all the other algorithms can just run on top of that.

• An important point is that the TPU uses 8 bit multiplication, which can be implemented much more efficiently than wider multiplication offered by the CPU. Such a low precision is sufficient and allows to pack many thousands of such multipliers on a single chip. – maaartinus Mar 27 '18 at 1:10

I think the algorithm has changed minimally, but the necessary hardware has been trimmed to the bone.

The number of gate transitions are reduced (perhaps float ops and precision too), as are the number of data move operations, thus saving both power and runtime. Google suggests their TPU achieves a 10X cost saving to get the same work done.

• ASIC - ASIC can be anything a GPU, CPU or a processor of your design, with any amount of memory you want to give to it. Let' say you want to design your own specialized ML processor, design a processor on ASIC. You want 256 bit FP number? Create 256 bit processor. You want your summing to be fast? Implement parallel adder upto a higher number of bits than conventional processors? You want n number of cores? No problem. You want to define the data-flow from different processing units to different places? You can do it. Also with careful planning you can get a trade-off between ASIC area vs power vs speed. The only problem is that for all of this you need to create your own standards. Generally some well defined standards are followed in designing processors, like number of pins and their functionality, IEEE 754 standard for floating point representation, etc which have been come up after lots of trial and errors. So if you can overcome all of these you can easily create your own ASIC.