I wanted to plot a graph to show the effect of increasing the batch size on loss calculated (MNIST dataset). But I am not able to decide if I should show change in loss over training time of the neural network or number of updates made to weights and biases (iterations and epochs basically, but for large differences in batch sizes, I think number of updates made makes more sense?). I am confused about what makes more sense (or neither of them makes sense, idk). enter image description here

With Loss vs training time graph, I can show that for any training time, the loss for large batch is more. From what I have read on wiki, with Loss vs number of updates made graph, I can show that change in loss is smoother for larger batches (rate of convergence). But can't the same conclusion be made when plotted against time? (Smooth convergence means lesser spikes right?)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.