0
$\begingroup$

There is a lot of discussion on google search about AI-custom-accelerators (like Intel's Gaudi) and GPUs.

Almost all of them say generic things like, a) AI Accelerator chip is for specialized AI processing whereas GPUs work for general AI models, and like b) if you want customize chip for specific AI workloads use AI-accelerator or else use GPU.

From what I understand GPUs are already great at large-scale dot-products done in batch-processing (throughput mode), and most of AI workloads are matmuls (which is essentially dot-product) so GPUs handle AI workloads very well.

Plus, I've also seen Intel's Gaudi being used for a "variety of AI workloads", not specialized for a single model. It can be used for general AI workloads just like GPU. So what's the difference.

What I don't understand is, "exactly" what specific features are built differently in Accelerator vs GPU. Both have ALUs and matmul engines that do very well on AI models. Both have large-cache/memory and DDR speed.

  • What exactly makes one better? For which AI workload would one choose accelerator over GPU?

  • AI accelerators have fixed-function for matmul. Do GPUs have fixed-function?

  • AI accelerators have software-managed cache (HBM) from what I understand. Is that the same with GPUs or is there a way cache is different between accelerators and GPUs that changes things?

I'm kind of unsure about the differences between GPUs and AI-accelerators with respect to Fixed-function and software-managed-cache.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

The fixed function part is what is different in different chips. Think of it like this, A GPU can calculate many different graphics related equations (textures, shaders, 3d Models, etc.) quickly, but an ASIC made specifically for image recognition will be much faster at recognizing images while not being able to render 3d models at all.

It is the difference between being able to solve many problems sort of quickly or being amazing at doing 1 specific task. A race car is very fast but sucks at transporting heavy loads. A Truck is slow but can move a lot of cargo. A Pickup truck is neither as fast as a race car, nor carries as much as an 18 wheeler, but it is pretty good at carrying medium loads, while being much faster than an 18 wheeler. A GPU is the pickup truck while the Asic is the race car or 18 wheeler depending on the task it is made for.

Improve this answer
$\endgroup$
1
$\begingroup$

Take a look here for a pretty good explanation from google on how their TPUs work. TPUs are ASICs as well as they are made with a very specific task in mind. They are way faster than GPUs for Neural Networks inference, but on the other side, a lot of model architectures are really hard to implement on them.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .