I need to train instance segmentation models on several different datasets. The datasets vary widely in how many instances each image contains.
For example:
Dataset 1: 100 images, each image has about 0 or 1 instances
Dataset 2: 100 images, each image has ~400 instances
Should I train for more iterations when learning on dataset 2? I was trying to reason about this in terms gradient updates. I'm using Mask R-CNN, and if I understand correctly there is only 1 gradient update per iteration, and the loss for a given image is the mean loss across each proposal.
loss for minibatch = sum(mean(loss for each proposal in given image) for each image in the minibatch)
(^ please correct me if this is wrong!)
So with more instances per image, the loss from each instance will get diluted, and so I'm thinking I should train for more iterations. On the other hand, I can see the argument that having more instances which each contribute proportionally less to the overall loss naturally balances everything out, and the model should train for the same number of iterations.
I tried searching around and couldn't find any papers or posts about this.
