1
$\begingroup$

While I have limited resource usually to train my machine learning models, I often find that my hyperparameter optimization procedure is not necessary using all my GPU and CPU, and that is because the results also depend on the batch size in my experience.

If you find in your project that a low batch size is necessary, how do you scale your project? In a multi-GPU scenario, I could imagine running different hyperparameter settings on different GPUs, but what other options are out there?

$\endgroup$
  • $\begingroup$ Hi and welcome to this community. It is not clear what you're asking. If you find in your project that a low batch size is necessary, how do you scale your project? What do you mean by "scale your project"? What is the relation between low batch size and scalability? IMHO, it is not very clear, in general, what you're asking. Are asking if there are any alternatives to hyper-parameter optimisation that runs on different GPUs? If yes, I could simply say: run the hyper-parameter optimisation using only one GPU. However, this is not what you're looking for, so be more specific and precise. $\endgroup$ – nbro Jul 26 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.