-2
$\begingroup$

Update 2

The OS I'm using is Windows 10, since we have WSL, I also use Ubuntu to run the code. The code is written in Python. I know there are thousands of factors which affect the final performance for model training, let's put it aside.

The scope of this post is to limited to talk about the very specific 2 CPUs: ThreadRipper 2920X and Ryzen 7 3700X in terms of performance for AI model training. Nothing more than this.

Update 1 for more information about what I am trying to achieve.

The initial motivation for this question: My mom has a fish pond in her backyard. Throughout years she found that birds come and stole fish and none of fake owl standing besides pond works, so she ends up covering the pond with a giant and ugly net which destroys all good sights in the backyard.

I need to train a model which recognizes birds, full stop and as simple as this.

So I have huge amount of data downloaded. How big? it contains 11K bird images for training, 5K for validation and 3K for testing.

For those who said a GPU is not needed, I agree. A cpu is not needed for some cases, but definitely not in my case. Why? coz I have started running my training since Saturday 2 weeks ago for 30 epochs. Since then it's been more than 1 week and only 12 epochs have been done. So I definitely need a GPU.

But let's get back to the topic. This topic is not about GPU, this topic is about comparing very specific CPUs for the sake of ML in terms of performance.

The CPUs mentioned here are AMD ThreadRipper 2920X Gen2 VS Ryzen 7 3700X

First of all, I know GPU matters much more than CPU and I have nailed down my GPU to be NVIDIA 2080 TI, so not asking for that.

I read several articles and found this one really helpful. In this article, it recommends to use AMD ThreadRipper 2920X and here is the build. But I wend to UserBenchMark and found Ryzen 7 3700X actually is newer, cheaper and performs better in most aspects.

My question is, should I keep everything same but just replace Threadripper 2920X with Ryzen 7 3700X? or should I stick to TR 2920X? and Why?

Improve this question
$\endgroup$
10
  • $\begingroup$ Your question does not even mention what kind of machine learning problem you want to tackle. Explain more the volume of data involved, and its kind. In many cases, a GPU is not even needed. Machine learning has been done since at least the 1990s (and actually decades before), at a time where GPUs did not even exist. I don't think that always putting most of your money in GPU is a wise decision (since RAM or high-speed SSD disks could matter more) $\endgroup$
    – Basile Starynkevitch
    Sep 3 '19 at 13:17
  • $\begingroup$ In other words, without a lot of additional details about your goals, the answer is always : it depends !! See also norvig.com/21-days.html for more. Your question is analog to "what is the best programming language" and that question has a similar answer: it depends .... e.g. you might look into SBCL; see also this answer $\endgroup$
    – Basile Starynkevitch
    Sep 3 '19 at 13:22
  • $\begingroup$ .... For example, I am not sure at all that TensorFlow or Gudhi always require more GPU and less CPU or RAM. I believe they don't. And the hardware resources they use depend on the problem you are trying to solve (not the same for real-time video and real-time audio machine learning techniques, or machine learning techniques for NLP or software engineering problems, such as static source code analysis, e.g. in DECODER $\endgroup$
    – Basile Starynkevitch
    Sep 3 '19 at 13:28
  • $\begingroup$ Feel free to send me an email to basile@starynkevitch.net (with the URL of your question) to discuss more these topics. See also some paragraphs of my often updated Bismon draft report mentioning future machine learning approaches. $\endgroup$
    – Basile Starynkevitch
    Sep 3 '19 at 13:34
  • $\begingroup$ What is your goal: learning to code, or making a complete software product (read about TRLs). $\endgroup$
    – Basile Starynkevitch
    Sep 3 '19 at 20:20
3
$\begingroup$

First of all, I know GPU matters much more than CPU

In general this is wrong or very naive. See also this answer (then consider SBCL) and that one (and also this and that).

The OS I'm using is Windows 10,

That is your biggest mistake, assuming you have access (and some understanding) to the source code of your application. See below why. Notice that the most expansive resource is your brainpower.

For example, you might deal with a large volume of data (e.g. a few terabytes, e.g. of post-processed raw HTML texts with NLP preprocessing in mind). In that case, SSD or RAM may matter much more than the GPU. See also this, but consider also any serious free software database management system (e.g. PostGreSQL, MongoDB, etc etc etc...)

On the contrary, in some applications, the volume of data is small, but its incoming bandwidth is very important (real-time video applications, à la autonomous vehicles).

In a comment, you mention bird recognition. Look into these slides and their mention of https://xkcd.com/1425/

Ask yourself: do you have hundred of gigabytes of bird photos? Tagged by humans? Do you want to process them quickly for an industrial application, or are you just starting to learn machine learning techniques. You'll need a decade to understand machine learning, but you could learn it with a cheap PC (or even a Raspberry Pi). You should take months or years to read books and papers (and wikipages).

Be aware that machine learning techniques can be tricked (see also this)

See also this (look who wrote that!)

My recommendation is to learn more about computer science, computer architecture, several programming languages, operating systems, algorithms, free software (e.g. TensorFlow and Ghudi ...), Linux.

If you code something which classifies successfully just a hundred thousands of bird images, going to ten million images is simply a matter of money. But learning to code that thing will take you months or years.

For learning and job opportunities purposes, investing more in books and less in hardware could be worthwhile, especially if you contribute to some existing open source software project (with your real name) on github or gitlab.

I used neural networks in the previous century on Sun workstations running some SunOS5, before even GPUs, as we know them today, existed. That (proprietary) code was on Unix, and today many free software or open source machine learning software exist (e.g. on github etc) and runs on most Linux PCs. Most of these libraries don't require GPUs, but some of them can take advantage of CUDA or of OpenCL (and OpenCL don't need a GPU, just take advantage of it when you enable it; an OpenCL code just runs a bit slower without GPUs, by a practical time factor of 10x to 1000x). Look also into OpenACC & MPI. You'll find free software (or at least gratis) implementations of all of them on Linux.

In a comment you also ask;

which CPU feeds the GPU better?

But that also depends not only on the executed code, but also on the data. And the data of your problem is, per se, problem specific.

In more concrete words, the same code (on Linux, e.g. using TensorFlow suitably configured to take advantage of your GPU) using OpenCL to drive the GPU will feed it in a rhythm which depends mostly of the data itself - not just of the code. Both GPU and CPU are involved, and that is exactly why OpenCL programming is so difficult. (I happened to try automatize that in a former OpenGPU project).

If you wish to solve a particular machine learning problem faster and already legally have the source code for that (running on Linux), consider also renting Cloud computing resources, and cloud computing is mostly used for machine learning purposes (read also about surveillance capitalism, since your question is at the heart of it). Most of them have fast GPGPUs. That is probably the cheapest and greenest solution for your concrete problem. Today, all cloud computers and every top500 supercomputer run on some variant of Linux. And cloud computing is counter-unintuitively greener than an equivalent PC box (at least in France, where electric power is mostly nuclear).

You mentioned a AMD ThreadRipper 2920X. I am sure that, every thing else being the same, my AMD 2970WX Linux workstation would solve your problem faster than your AMD 2920 based Linux system (because an 2970 has more cores!), with the same source code. And Linux (used in datacenters on the Cloud, in top500 supercomputers) is the most practically useful operating system for machine learning. For machine learning programming you should use Linux (not Windows) because it usually takes better advantage of multi-core machines, supercomputer, cloud computers.

The scope of this post is to limited to talk about the very specific 2 CPUs: ThreadRipper 2920X and Ryzen 7 3700X in terms of performance for AI model training. Nothing more than this.

Did you google that question: ThreadRipper 2920X vs Ryzen 7 3700X ? The following step is to use your intuition to find out, amongst the various benchmarks mentioned in answers, which one is the most similar to your code (assuming you understand it). Think also of energy costs.

The code is written in Python.

Are you aware that Python is a lot slower than C++? Of course, most of the computer time would be spent in numerical libraries (those coded also in OpenCL to use your GPUs). But the same algorithm coded in C++ (or even in Go) and in Python is a lot faster in C++ or Go. Depending on the problem, you could win just a few percents, or a factor of ten (be aware of Python's GIL and consider using Guile - you could be more productive in it than with Python. See SICP, still the best introduction to programming I ever read).

So I definitely need a GPU.

No, you need more computing power. And you could rent it and build then use some Web service above that using a cheap Wifi connection (even with a RaspBerryPi) to the Internet. Think also of IoT approaches to your (still unclear) problem. In most parts of Europe or USA, IoT approaches (combined with cloud computing) make a lot of economical sense. For pond surveillance, they probably are the cheapest. And I cannot imagine why pond surveillance is so confidential to forbid such a mixed (IoT + cloud computing) approach. Your pond surveillance problem is not safety-critical.

My question is, should I keep everything same but just replace Threadripper 2920X with Ryzen 7 3700X? or should I stick to TR 2920X? and Why?

That question is off-topic here. https://superuser.com/ could be a better place for it

PS. A few paragraphs of my continously evolving Bismon draft report, and several references there, the DECODER project, are very relevant to your question.

PPS. Read more about TRLs. For learning purposes, they don't matter a lot (but your own time does). For commercial or lucrative purposes, it is the main issue (and costs depend exponentially on the TRL level).

Improve this answer
$\endgroup$
17
  • $\begingroup$ The point is what is the goal of the OP. My understanding is that he wants to learn machine learning. For that a GPU is not even needed (but convenient). A necessary tool to learn machine learning is a human brain. $\endgroup$
    – Basile Starynkevitch
    Sep 3 '19 at 20:11
  • $\begingroup$ hi @BasileStarynkevitch I agree that we do not need a GPU to use NN and that's what I'm doing: running training model without GPU , only on CPU. But it is super slow. My training will take 17 days for 30 epochs. That's why I purchased my new PC box. The topic is to discuss about the choice of CPU not the academic stuff. $\endgroup$
    – Franva
    Sep 4 '19 at 2:09
  • $\begingroup$ Why did you not consider renting some cloud computing resources. They are probably cheaper in your case, and more efficient. Many cloud computers have GPUs. $\endgroup$
    – Basile Starynkevitch
    Sep 4 '19 at 4:09
  • $\begingroup$ Do you have the source code, or are you writing it? If you can recompile source code, the cheapest solution is renting cloud computing resources (the trade-off is less confidentiality, but it mostly concerns the data, not the code: decompilation is not cost-effective) $\endgroup$
    – Basile Starynkevitch
    Sep 4 '19 at 4:29
  • $\begingroup$ If you don't have and are not writing the source code for your (still unclear) machine learning problem, that should be mentioned in your question $\endgroup$
    – Basile Starynkevitch
    Sep 4 '19 at 4:32
2
$\begingroup$

Usually, the only relevant criterion is whether the CPU can feed the GPU fast enough. How much preprocessing does your task require?

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ The answer is correct but lacks details and thus more suitable as a comment. $\endgroup$
    – user9947
    Sep 3 '19 at 7:25
  • $\begingroup$ All I need to do is training bird recognition with Tensorflow or Py Torch. Sorry I do not know much about it as I just started in AI. $\endgroup$
    – Franva
    Sep 3 '19 at 7:47
  • $\begingroup$ @Franva: Bird detection? I know Queen Mary University of London works on audio recognition, while other groups do video recognition. That has quite an impact on the data size, and thus the CPU load. $\endgroup$
    – MSalters
    Sep 3 '19 at 8:42
  • $\begingroup$ @MSalters yep, for me it's just recognize birds from an image or video stream. It will be very happy if I could achieve this goal :) $\endgroup$
    – Franva
    Sep 3 '19 at 9:26
  • $\begingroup$ hi @MSalters could you please elaborate how to calculate whether a CPU can feed GPU fast enough??? Let's take my case as a concrete example: I have my GPU NVIDIA RTX 2080TI 11GB, which CPU feeds it quicker, ThreadRipper 2920X or Ryzen 7 3700X? $\endgroup$
    – Franva
    Sep 4 '19 at 3:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.