Is there any guidance available for training on very noisy data, when Bayes error rate (lowest possible error rate for any classifier) is high? For example, I wonder if deliberately (not due to memory or numerical stability limitations) lowering the batch size or learning rate could produce a better classifier.

I found so far some general recommendations, not specific for noisy data: Tradeoff batch size vs. number of iterations to train a neural network

  • 1
    $\begingroup$ Shouldn't it be the contrary? Batch learning is used as it cancels out noise due to all the samples taken...Whereas in the extreme case of batch Learning where batch size = 1 also called online learning results in very shaky convergence with spikes in loss. $\endgroup$
    – user9947
    Feb 22 '19 at 9:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.