0
$\begingroup$

I have about 2000 items in my validation set, would it be reasonable to calculate the loss/error after each epoch on just a subset instead of the whole set, if calculating the whole dataset is very slow?

Would taking random mini-batches to calculate loss be a good idea as your network wouldn't have a constant set? Should I just shrink the size of my validation set?

$\endgroup$
1
  • $\begingroup$ I would also suggest that you explain a little bit the model you're using, how much time it takes to compute the validation loss and what is the task you're trying to solve. $\endgroup$ – nbro Feb 15 at 12:24
1
$\begingroup$

I assume you intended to write compute the evaluation metric over the validation set in batches; you do not compute loss over the validation set!

That is quite a standard practice in many academic implementations (because, when the validation set is large enough, the memory will be a constraint), however, be sure to take the average of the values over all the batches. Using a K-fold setup will increase the confidence in the reported values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.