Hot answers tagged

11 votes
Accepted

Effect of batch size and number of GPUs on model accuracy

This should make a difference, but how big is the difference heavily depends on your task. However, generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but ...
Clement's user avatar
  • 1,735
9 votes

Can LSTM neural networks be sped up by a GPU?

From Nvidia www (https://developer.nvidia.com/discover/lstm): Accelerating Long Short-Term Memory using GPUs The parallel processing capabilities of GPUs can accelerate the LSTM training and ...
pasaba por aqui's user avatar
7 votes
Accepted

Is a GPU always faster than a CPU for training neural networks?

This changes according to your data and complexity of your models. See following article by microsoft. Their conclusion is The results suggest that the throughput from GPU clusters is always ...
Atilla Ozgur's user avatar
6 votes
Accepted

Can LSTM neural networks be sped up by a GPU?

I found that there are cuDNN accelerated cells in Keras, for example, https://keras.io/layers/recurrent/#cudnnlstm. They are very fast. The normal LSTM cells are faster on CPU than on GPU.
Dieshe's user avatar
  • 289
4 votes
Accepted

What is accelerated years in describing the amount of the training time?

The statement in which you mentioned that "GPT-3 took 405 V100 years to train" refers to the computational resources utilized in training the GPT-3 model. Specifically measured in terms of ...
Keval's user avatar
  • 111
4 votes

Is it true that batch size of form $2^k$ gives better results?

The choice of the batch size to be a power of 2 is not due the quality of predictions . The larger the batch_size is - the better is the estimate of the gradient, but a noise can be beneficial to ...
spiridon_the_sun_rotator's user avatar
3 votes
Accepted

Choosing proper graphic card for deep learning AND gaming

All cards from this series support CUDA. In fact they even have special cores, designed for faster deep learning calculations called 'tensorcores'. If you want to do some deep learning with big models ...
GKozinski's user avatar
  • 1,250
3 votes

Should I spend money on a machine-learning capable PC or just use Google CoLab?

To the day, I have found 3 places with free GPU: Colab, Kaggle and gradient.run (haven't tested the last one yet). There are several issues with all of them: Usually very weak CPU (even my 10-y.o. ...
Melanol's user avatar
  • 93
3 votes
Accepted

How do GPUs faciliate the training of a Deep Learning Architecture?

GPUs are able to execute a huge amount of similar and simple instructions (floating point operations like addition and multiplication) in parallel. In contrast to a CPU which is able to execute a few ...
Tinu's user avatar
  • 628
3 votes

Effect of batch size and number of GPUs on model accuracy

No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely ...
Ketil Malde's user avatar
2 votes

Is a GPU always faster than a CPU for training neural networks?

I advice you to always use GPU over CPU for training your models. This is driven by the usage of deep learning methods on images and texts, where the data is very rich. You must have a GPU suited ...
Varshitha N S's user avatar
2 votes

How does a transformer leverage the GPU to be trained faster than RNNs?

A recurrent neural network (RNN) depends on the previous hidden state from the previous time step. That is, an RNN is a function of both the data for the sequence at time $t$ and the hidden state from ...
David's user avatar
  • 4,760
2 votes
Accepted

Training network with 4 GPUs performance is not exactly 4 times over one GPU why?

Your dataset class probably have a lot of preprocessing code. You should use a dataloader. It will prefetch data from the dataset when the GPU is processing. Also, you can process all the data ...
Clement's user avatar
  • 1,735
2 votes
Accepted

How can I reduce the GPU memory usage with large images?

As the other links suggest, you basically have four options: "restrict your CNN". This means making your model smaller and simpler, possibly by inserting a pooling layer at the front, or reducing the ...
John Doucette's user avatar
2 votes

How are gaming GPUs different from GPUs for ML?

Usually the difference is mainly about drivers and amount of VRAM (other than price..). For example, high-end NVIDIA GPUs tend to even share the same architecture, having the same amount of "...
Luca Anzalone's user avatar
2 votes

For an LLM model, how can I estimate its memory requirements based on storage usage?

It really depends on what you want to do with it, how you load it and what else you add. For inference at half-precision (16 bit per param) a 7B param models should be 7 billion * 2 bytes per param or ...
Andrew Mellinger's user avatar
2 votes

For an LLM model, how can I estimate its memory requirements based on storage usage?

It varies depending on various factors such as quantization. My rough rule of thumb is memory need is 2-4x of the disk size. Just as an example, the model at https://huggingface.co/TheBloke/wizardLM-...
beejay's user avatar
  • 21
2 votes
Accepted

How to speed up my neural network?

There are a bunch of things that can influence how fast your code runs, not necessarily to do with the architecture but more programming wise. For example, you can have slow implementations of adding ...
Robin van Hoorn's user avatar
2 votes
Accepted

How much computing power does it cost to run GPT-3?

I can't anwser your question on how much computing power you might need, but you'll need atleast a smallgrid to run the biggest model just looking at the memory requirments (175B parameters so 700GB ...
hal9000's user avatar
  • 359
1 vote
Accepted

Has anyone tried to use llama.cpp with NVLink?

memory pooling is not really much of a thing these days: the interface is not really that all of a sudden you get a single address space. You still have individual GPUs, you just specify the ...
Steven Lu's user avatar
  • 126
1 vote

How "exactly" are AI-accelerator chip ASICs built differently than GPUs as GPU seem to lead for many AI workloads on performance

Take a look here for a pretty good explanation from google on how their TPUs work. TPUs are ASICs as well as they are made with a very specific task in mind. They are way faster than GPUs for Neural ...
Darius Chira's user avatar
1 vote

How "exactly" are AI-accelerator chip ASICs built differently than GPUs as GPU seem to lead for many AI workloads on performance

The fixed function part is what is different in different chips. Think of it like this, A GPU can calculate many different graphics related equations (textures, shaders, 3d Models, etc.) quickly, but ...
Excited Box's user avatar
1 vote

Why training the same model on the same data can be slower on better card?

Every training will be slightly different because of the statistical matter of neural networks. The question is, how much is your better? Then, newer does not imperatively mean better. It means more ...
Skobo Do's user avatar
1 vote

What kind of neural network and GPU should I use to classify images into > 10 000 classes?

You could look for papers that trained models on the Open Image Dataset, which contains around 6k classes, so pretty close to your final use case. Regarding the dataset size, most datasets include at ...
Edoardo Guerriero's user avatar
1 vote

Is it true that batch size of form $2^k$ gives better results?

The main reason to use powers of 2 is in the way existing hardware and software are made, there isn't any purely mathematical reason. CPUs, GPUs, memories, and internal buses all use a size that's the ...
cactusresistance's user avatar
1 vote

How much computing power does it cost to run GPT-3?

I think it is premature to answer your question as OpenAI has not made GPT-3 available yet other than via a web-based API. For more information see OpenAI API. From OpenAI will start selling its text-...
Brian O'Donnell's user avatar
1 vote

Is a GPU always faster than a CPU for training neural networks?

It depends, if you ve to solve a "simple" problem which does not require CNN or stacked models without multidimensional data and not many multiplications, big long numbers then if you decide to use ...
nikolaosmparoutis's user avatar
1 vote

Does fp32 & fp64 performance of GPU affect deep learning model training?

Deep models are very tolerant to arithmetic underflow. You can hope for neglectable differences in prediction accuracy between FP32 and FP16 models. Check this paper for concrete results.
ssegvic's user avatar
  • 499

Only top scored, non community-wiki answers of a minimum length are eligible