I ran my job on a computing cluster: first with 4 cores, then with 32 cores (2 nodes). But the training time is pretty much exactly the same for both of them: ~67 seconds per step.

I am trying to fine-tune GPT-2 for my text dataset (chat logs).

What can I do to get a performance increase with to the increase in processing power with CPUs?

  • $\begingroup$ Have you tried all these? - Adjusting nodes - Increase Threads - Increase Cache sizes - Over clock the CPU's (not advisable) $\endgroup$ – Michael Hearn Nov 10 '19 at 19:17
  • $\begingroup$ This question is off topic for this site, but I think the default thread count used in tensorflow is quite small (3?), so if you do not set this higher, running on a 32-core machine will not be faster than running on a 3-core or 4-core machine. $\endgroup$ – John Doucette Nov 11 '19 at 14:34
  • $\begingroup$ @JohnDoucette How would I do that? I see similar questions about GPU/CPU performance so it seemed on topic. $\endgroup$ – animehistrionics Nov 13 '19 at 3:53
  • $\begingroup$ It's something you can set as a configuration parameter in tensor flow, during the initialization setup iirc $\endgroup$ – John Doucette Nov 13 '19 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.