Why should data batches in a neural network have an equal size?
I have seen some recent research works on making the batch size dynamic, but still, I can't find an answer to my question.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
They don't have to be equal, but introducing variable batches brings more complexity.
In particular you need to make sure that batches are representative enough of the full dataset. If the batches have the same size you can just test few batch values until you get satisfactory results. If you introduce variable batches you need to perform more complicated test on the distribution/rule controlling this variability and handle edge cases, which is an additional headache over an already complicated debugging process. Usually it's not worth the pain, there are more impactful ways of spending your development time.
