If one examines the SSD: Single Shot MultiBox Detector code from this GitHub repository, it can be seen that, for a testing phase (evaluating network on test data set), there is a parameter test batch size. It is not mentioned in the paper.
I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?
I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?
It is usually just for memory use limitation vs speed of assessment. Larger batches evaluate faster on parallelised systems such as GPUs, but use more memory to process. Test results should be identical, with same size of dataset and same model, regardless of batch size.
Typically you would set batch size at least high enough to take advantage of available hardware, and after that as high as you dare without taking the risk of getting memory errors. Generally there is less to gain than with training optimisation though, so it is not worth spending a huge amount of time optimising the batch size to each model you want to test. In most code I have seen, users pick a moderate "safe" value that will speed up testing but doesn't risk failing if you wanted to add a few layers to the model and check what that does.
