9
$\begingroup$

Currently, I am working on a few projects that use feedforward neural networks for regression and classification of simple tabular data. I have noticed that training a neural network using TensorFlow-GPU is often slower than training the same network using TensorFlow-CPU.

Could something be wrong with my setup/code or is it possible that sometimes GPU is slower than CPU?

Improve this question
$\endgroup$
0

3 Answers 3

Reset to default
7
$\begingroup$

This changes according to your data and complexity of your models. See following article by microsoft. Their conclusion is

The results suggest that the throughput from GPU clusters is always better than CPU throughput for all models and frameworks proving that GPU is the economical choice for inference of deep learning models. ...

It is important to note that, for standard machine learning models where number of parameters are not as high as deep learning models, CPUs should still be considered as more effective and cost efficient.

Since you are training MLP, it can not be thought as standard machine learning model. See my preprint, The impact of using large training data set KDD99 on classification accuracy. I compare different machine learning algorithms using weka.

enter image description here

As you can see from above image, MLP takes 722 minutes to train while Naive Bayes ~2 minutes. If your data is small and your models parameters are not high, you will see better performance on CPU.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ You are correct. I run some tests and found out that the networks that are trained faster on GPU are these that are bigger (more layers, more neurons). Smaller ones are faster on CPU. Thanks for clearing this out! $\endgroup$
    – GKozinski
    Commented Aug 25, 2019 at 15:10
  • $\begingroup$ Also worth noting that some network organizations may not be well suited to operating on GPUs and CPU may be faster for those. E.g. cascade correlation networks, where each node can depend on the output of any other node with a lower node number, so it is essentially impossible to calculate them in parallel. $\endgroup$
    – occipita
    Commented Feb 11, 2023 at 21:25
2
$\begingroup$

I advice you to always use GPU over CPU for training your models. This is driven by the usage of deep learning methods on images and texts, where the data is very rich.

You must have a GPU suited perfectly for training (e.g. NVIDIA 1080, NVIDIA Titan or higher versions), I wouldn't be surprised to find that your CPU was faster if you don't have a powerful GPU.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Yeah, I forgot to mention my gpu is gtx 970, cpu is xeon e-3 1220. Like I said I only train numerical tabular data, no images or text. $\endgroup$
    – GKozinski
    Commented Aug 24, 2019 at 18:45
1
$\begingroup$

It depends, if you ve to solve a "simple" problem which does not require CNN or stacked models without multidimensional data and not many multiplications, big long numbers then if you decide to use CNN / stacked architectures AND GPU it is like using a hammer to insert a needle.It will not only spend energy but the computations will do zero padding in memory, you will observe a degradation in speed.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .