0
$\begingroup$

I have about 2000 items in my validation set, would it be reasonable to calculate the loss/error after each epoch on just a subset instead of the whole set, if calculating the whole dataset is very slow?

Would taking random mini-batches to calculate loss be a good idea as your network wouldn't have a constant set? Should I just shrink the size of my validation set?

Improve this question
$\endgroup$
1
  • $\begingroup$ I would also suggest that you explain a little bit the model you're using, how much time it takes to compute the validation loss and what is the task you're trying to solve. $\endgroup$
    – nbro
    Commented Feb 15, 2021 at 12:24

1 Answer 1

Reset to default
1
$\begingroup$

I assume you intended to write compute the evaluation metric over the validation set in batches; you do not compute loss over the validation set!

That is quite a standard practice in many academic implementations (because, when the validation set is large enough, the memory will be a constraint), however, be sure to take the average of the values over all the batches. Using a K-fold setup will increase the confidence in the reported values.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .