11
votes
Accepted
Effect of batch size and number of GPUs on model accuracy
This should make a difference, but how big is the difference heavily depends on your task. However, generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but ...
9
votes
Can LSTM neural networks be sped up by a GPU?
From Nvidia www (https://developer.nvidia.com/discover/lstm):
Accelerating Long Short-Term Memory using GPUs
The parallel processing capabilities of GPUs can accelerate the LSTM training and ...
7
votes
Accepted
Is a GPU always faster than a CPU for training neural networks?
This changes according to your data and complexity of your models. See following article by microsoft. Their conclusion is
The results suggest that the throughput from GPU clusters is always
...
6
votes
Accepted
Can LSTM neural networks be sped up by a GPU?
I found that there are cuDNN accelerated cells in Keras, for example, https://keras.io/layers/recurrent/#cudnnlstm.
They are very fast. The normal LSTM cells are faster on CPU than on GPU.
4
votes
Is it true that batch size of form $2^k$ gives better results?
The choice of the batch size to be a power of 2 is not due the quality of predictions .
The larger the batch_size is - the better is the estimate of the gradient, but a noise can be beneficial to ...
4
votes
Accepted
What is accelerated years in describing the amount of the training time?
The statement in which you mentioned that "GPT-3 took 405 V100 years to train" refers to the computational resources utilized in training the GPT-3 model. Specifically measured in terms of ...
3
votes
Accepted
Has anyone tried to use llama.cpp with NVLink?
memory pooling is not really much of a thing these days: the interface is not really that all of a sudden you get a single address space. You still have individual GPUs, you just specify the ...
3
votes
Accepted
Choosing proper graphic card for deep learning AND gaming
All cards from this series support CUDA. In fact they even have special cores, designed for faster deep learning calculations called 'tensorcores'.
If you want to do some deep learning with big models ...
3
votes
Should I spend money on a machine-learning capable PC or just use Google CoLab?
To the day, I have found 3 places with free GPU: Colab, Kaggle and gradient.run (haven't tested the last one yet). There are several issues with all of them:
Usually very weak CPU (even my 10-y.o. ...
3
votes
Accepted
How do GPUs faciliate the training of a Deep Learning Architecture?
GPUs are able to execute a huge amount of similar and simple instructions (floating point operations like addition and multiplication) in parallel. In contrast to a CPU which is able to execute a few ...
3
votes
Effect of batch size and number of GPUs on model accuracy
No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely ...
2
votes
Is a GPU always faster than a CPU for training neural networks?
I advice you to always use GPU over CPU for training your models. This is driven by the usage of deep learning methods on images and texts, where the data is very rich.
You must have a GPU suited ...
2
votes
How does a transformer leverage the GPU to be trained faster than RNNs?
A recurrent neural network (RNN) depends on the previous hidden state from the previous time step. That is, an RNN is a function of both the data for the sequence at time $t$ and the hidden state from ...
2
votes
Accepted
Training network with 4 GPUs performance is not exactly 4 times over one GPU why?
Your dataset class probably have a lot of preprocessing code. You should use a dataloader. It will prefetch data from the dataset when the GPU is processing. Also, you can process all the data ...
2
votes
Accepted
How can I reduce the GPU memory usage with large images?
As the other links suggest, you basically have four options:
"restrict your CNN". This means making your model smaller and simpler, possibly by inserting a pooling layer at the front, or reducing the ...
2
votes
How are gaming GPUs different from GPUs for ML?
Usually the difference is mainly about drivers and amount of VRAM (other than price..).
For example, high-end NVIDIA GPUs tend to even share the same architecture, having the same amount of "...
2
votes
For an LLM model, how can I estimate its memory requirements based on storage usage?
It really depends on what you want to do with it, how you load it and what else you add.
For inference at half-precision (16 bit per param) a 7B param models should be 7 billion * 2 bytes per param or ...
2
votes
For an LLM model, how can I estimate its memory requirements based on storage usage?
It varies depending on various factors such as quantization. My rough rule of thumb is memory need is 2-4x of the disk size. Just as an example, the model at https://huggingface.co/TheBloke/wizardLM-...
2
votes
Accepted
How to speed up my neural network?
There are a bunch of things that can influence how fast your code runs, not necessarily to do with the architecture but more programming wise. For example, you can have slow implementations of adding ...
2
votes
Accepted
How much computing power does it cost to run GPT-3?
I can't anwser your question on how much computing power you might need, but you'll need atleast a smallgrid to run the biggest model just looking at the memory requirments (175B parameters so 700GB ...
1
vote
Why gpu works faster for float number rather then zeros?
Due to their architecture, GPUS is faster on floating numbers rather than integers and binary numbers like one and zero. they are designed to perform floating point arithmetic operations such as ...
1
vote
How "exactly" are AI-accelerator chip ASICs built differently than GPUs as GPU seem to lead for many AI workloads on performance
Take a look here for a pretty good explanation from google on how their TPUs work. TPUs are ASICs as well as they are made with a very specific task in mind. They are way faster than GPUs for Neural ...
1
vote
How "exactly" are AI-accelerator chip ASICs built differently than GPUs as GPU seem to lead for many AI workloads on performance
The fixed function part is what is different in different chips. Think of it like this, A GPU can calculate many different graphics related equations (textures, shaders, 3d Models, etc.) quickly, but ...
1
vote
Why training the same model on the same data can be slower on better card?
Every training will be slightly different because of the statistical matter of neural networks.
The question is, how much is your better?
Then, newer does not imperatively mean better. It means more ...
1
vote
What kind of neural network and GPU should I use to classify images into > 10 000 classes?
You could look for papers that trained models on the Open Image Dataset, which contains around 6k classes, so pretty close to your final use case.
Regarding the dataset size, most datasets include at ...
1
vote
Is it true that batch size of form $2^k$ gives better results?
The main reason to use powers of 2 is in the way existing hardware and software are made, there isn't any purely mathematical reason. CPUs, GPUs, memories, and internal buses all use a size that's the ...
1
vote
How much computing power does it cost to run GPT-3?
I think it is premature to answer your question as OpenAI has not made GPT-3 available yet other than via a web-based API. For more information see OpenAI API.
From OpenAI will start selling its text-...
1
vote
Is a GPU always faster than a CPU for training neural networks?
It depends, if you ve to solve a "simple" problem which does not require CNN or stacked models without multidimensional data and not many multiplications, big long numbers then if you decide to use ...
Only top scored, non community-wiki answers of a minimum length are eligible
Related Tags
gpu × 42training × 12
deep-learning × 10
neural-networks × 8
tensorflow × 7
pytorch × 6
hardware × 6
machine-learning × 5
large-language-models × 5
convolutional-neural-networks × 4
keras × 4
inference × 3
computer-vision × 2
python × 2
long-short-term-memory × 2
transformer × 2
image-processing × 2
attention × 2
batch-size × 2
hardware-evaluation × 2
memory × 2
reinforcement-learning × 1
natural-language-processing × 1
image-recognition × 1
optimization × 1