0
$\begingroup$

I am training a very deep neural network (Panoptic-DeepLab) with a ResNet34 backbone on Google Colab on CityScapes dataset for Panoptic Segmentation, and noticed that, with a big crop size, the batch size has to be decreased to 1 image per batch, otherwise CUDA out of memory issues start to occur. While I know that this can create skewness in the training and it will likely be very hard to attain good convergence, can I ask this question in general to the experts: how valid is a batch size of 1 generally considered in image-based processing? The images in consideration can be considered large (high resolution). The optimizer used is Adam alongwith a warm up polynomial learning rate (with base around 0.00005), and 90k iterations.

(I understand that it would possibly be a good idea to try out a smaller crop size and bigger batch size, but would like to know the feedback from the community anyway)

Improve this question
$\endgroup$
1
  • $\begingroup$ You can do gradient accumulation to artificially use a higher batch size even if only one image can fit in the memory. $\endgroup$
    – Lelouch
    Commented Sep 1, 2022 at 8:12

1 Answer 1

Reset to default
1
$\begingroup$

After more research, I found that a batch size of 1 is quite common in deep-learning image processing use-cases where there are high memory/GPU requirements for model training. In fact, it gives better results at times.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Commented Jul 19, 2022 at 16:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .