How are gaming GPUs different from GPUs for ML? [closed]

Question

I have a seemingly innocent question. I was looking to buy a computer for deep learning development using large language models. When I searched online on consumer e-commerce websites, I could not find any computer with NVIDIA P100, V100 for sale. Why? These are the GPUs used for deep learning on cloud VMs. Consumer GPUs (I use this term to mean GPUs available in computers you will find on sale on e-commerce websites such as Amazon.com) on the other hand seem to be centered around so-called RTX series built for gaming applications. I tried looking for GPU benchmarks on 3 different sites [1,2,3] and cannot find any mention of P100, V100 etc. All of them are filled with RTX.

What is the difference between these chipsets? And more specifically, is there some inherent difference that will make RTX GPUs suboptimal if I were to use one for ML tasks (as opposed to gaming)?

Answer 1

Usually the difference is mainly about drivers and amount of VRAM (other than price..).

For example, high-end NVIDIA GPUs tend to even share the same architecture, having the same amount of "compute units", between RTX and A series (or the older P and V). The difference is that professional GPUs have more optimized drivers which are both more stable and faster for supported applications.

Another difference is that the A series (e.g. A6000 or A100) tend to have way larger VRAM capacity (e.g. 48GB and even 80GB) which are more suitable to handle large models, and even large batch sizes during training.

In principle, when picking a GPU for DL you should care more about VRAM than execution speed.