# Using batches in testing

If one examines SSD: Single Shot MultiBox Detector code from GitHub repository, it can be seen that, for a testing phase (evaluating network on test data set), there is a parameter test batch size. It is not mentioned in the paper.

I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?

I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?

It is usually just for memory use limitation vs speed of assessment. Larger batches evaluate faster on parallelised systems such as GPUs, but use more memory to process. Test results should be identical, with same size of dataset and same model, regardless of batch size.

Typically you would set batch size at least high enough to take advantage of available hardware, and after that as high as you dare without taking the risk of getting memory errors. Generally there is less to gain than with training optimisation though, so it is not worth spending a huge amount of time optimising the batch size to each model you want to test. In most code I have seen, users pick a moderate "safe" value that will speed up testing but doesn't risk failing if you wanted to add a few layers to the model and check what that does.

Test batch time is referenced in the paper on page 33, Section 3.6, Inference Time and in Table 6. When an example set is used, a portion of it is used for training and another portion is used for testing. In mini-batch configurations the mini batch sizes are also independently configurable between train and test phases in many implementations.

References

SSD: Single Shot MultiBox Detector, 2016, Wei Liu

• Thanks for your answer but still I do not understand, how/why is it used? Just to speed up execution time? – carobnodrvo Jan 27 '19 at 13:09