2022 Developer Survey is open! Take survey.

Questions tagged [gpu]

Graphics processing units or GPUs are specialized hardware for the manipulation of images and calculation of local image properties.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

What is the ideal GPU/CPU requirment for albert xxlargev1 squad2?

I am quite new to AI and Deep learning but I have specific question about what is the requiments to speed/create a conversational AI based on ALbert model. https://huggingface.co/ahotrod/...
user avatar
0 votes
2 answers
90 views

Is it true that batch size of form $2^k$ gives better results?

I am confused among the following in selecting the batch size for my model. #1: powers of 2 I generally see that batch sizes are in powers of two: 32, 64, 128, 256. #2: maximum GPU Suppose my GPU ...
user avatar
  • 3,029
0 votes
0 answers
38 views

Are there any inference memory requirement tables for Hugging Face transformers?

Hugging Face has a very large list of supported transformers. They provide a table which gives the status on whether or not a transformer has a slow tokenizer, a fast tokenizer, PyTorch support, ...
user avatar
2 votes
0 answers
64 views

How to train neural networks with multiprocessing?

I am trying to figure out how multiprocessing works in neural networks. In the example I've seen, the database is split into $x$ parts (depending on how many workers you have) and each worker is ...
user avatar
1 vote
0 answers
22 views

How can traditional edge detection algorithms be implemented on a GPU?

How can edge detection algorithms, which are not based on deep learning, such as the canny edge detector, be implemented on a GPU? For example, how are non-edge pixels removed from an image once it ...
user avatar
1 vote
2 answers
2k views

How much computing power does it cost to run GPT-3? [closed]

I know it cost around $4.3 million dollars to train, but how much computing power does it cost to run the finished program? IBM Watson chatbot AI only costs a few cents per chat message to use, ...
user avatar
  • 121
2 votes
1 answer
100 views

How do GPUs faciliate the training of a Deep Learning Architecture?

I would love to know in detail, how exactly GPUs help, in technical terms, in training the deep learning models. To my understanding, GPUs help in performing independent tasks simultaneously to ...
user avatar
8 votes
2 answers
3k views

Effect of batch size and number of GPUs on model accuracy

I have a data set that was split using a fixed random seed and I am going to use 80% of the data for training and the rest for validation. Here are my GPU and batch size configurations use ...
user avatar
0 votes
0 answers
24 views

What are the properties of a model that is well suited for for high performance real-time inference

What are general best practices or considerations in designing a model that is optimized for real-time inference?
user avatar
3 votes
0 answers
26 views

Good model and training algorithm to store texture data for fast gpu inference

Now, the following may sound silly, but I want to do it for my better understanding of performance and implementation of GPU inference for a set of deep learning problems. What I want to do is to ...
user avatar
4 votes
2 answers
295 views

How does a transformer leverage the GPU to be trained faster than RNNs?

How does a transformer leverage the GPU to be trained faster than RNNs? I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the ...
user avatar
  • 151
3 votes
1 answer
126 views

Training network with 4 GPUs performance is not exactly 4 times over one GPU why? [closed]

Training neural network with 4 GPUs using pyTorch, performance is not even 2 times (btw 1 & 2 times) compare to using one GPU. From Nvidia-smi we see GPU usage is for few milliseconds and next 5-...
user avatar
  • 73
3 votes
1 answer
1k views

How can I reduce the GPU memory usage with large images?

I am trying to train a CNN-LSTM model. The size of my images is 640x640. I have a GTX 1080 ti 11GB. I am using Keras with the TensorFlow backend. Here is the model. ...
user avatar
8 votes
3 answers
4k views

Is a GPU always faster than a CPU for training neural networks?

Currently, I am working on a few projects that use feedforward neural networks for regression and classification of simple tabular data. I have noticed that training a neural network using TensorFlow-...
user avatar
  • 1,027
2 votes
0 answers
22 views

In addition to matrix algebra, can GPU's also handle the various Kernel functions for Neural Networks?

I've read a number of articles on how GPUs can speed up matrix algebra calculations, but I'm wondering how calculations are performed when one uses various kernel functions in a neural network. If ...
user avatar
1 vote
0 answers
42 views

Any good resources for learning programming GPU level operations? [closed]

I want to be able to improve my lower level device specific programming abilities to assist in future endeavors. Examples would be learning to write custom tensorflow operations in C++ optimized to ...
user avatar
  • 2,229
1 vote
1 answer
250 views

Can I calculate the training performance of GPUs by comparing their specification? [closed]

I am currently using Nvidia GTX1050 with 640 CUDA cores and 2GB GDDR5 for Deep Neural Network training. I want to buy a new GPU for training, but I am not sure how much performance improvement I can ...
user avatar
  • 43
1 vote
2 answers
4k views

Does fp32 & fp64 performance of GPU affect deep learning model training? [closed]

I am purchasing Titan RTX GPU. Everything seems fine with that except float32 & float64 performance which seems lower vis-a-vis some of its counter parts. I wanted to understand if single ...
user avatar
7 votes
2 answers
17k views

Can LSTM neural networks be sped up by a GPU?

I am training LSTM neural networks with Keras on a small mobile GPU. The speed on the GPU is slower than on the CPU. I found some articles that say that it is hard to train LSTMs (and, in general, ...
user avatar
  • 269