Can too much trainingdata have a negative impact?

Question

I have to detect objects in an image. I want to use a neural network for this (yolov8). Since my objects are stacked, most of them are partially hidden and only front and side is visible. My dataset for training contains 50% images where the objects are fully visible and 50% where the objects are stacked. In reality, only the top object of the stack is fully visible and there are 4-5 objects per stack, so maybe 25% of the detecable objects are fully visible and 75% are partially hidden.

Can it have a negative impact if I use all of the training data? So should I use only the stacked-images for training or is it always a good idea to use as much trainingdata as possible? Maybe the model can find some useful informations of the objects in the fully-visible-images. Or maybe too much fully visible objects are misleading?

Since the trainingdata isn't labeled yet I can't test both options. And I don't want to waste too much time in labeling useless data.

What are your thoughts on this? Thanks!

Answer 1

Yes, however, most likely not at your scale

you can read more here https://arxiv.org/abs/2001.08361 but in poor words, they show that the scaling of the parameters and the data are not orthogonal, and that smaller models, on bigger datasets, have worse performances than trained on smaller dataset

It's like they get overwhelmed of informations, and instead of becoming good at something, they get mediocre over everything (consider that this applies to LLMs, where they have to predict the next token, thus if you don't have enough memory to recall given a context what comes next, then you will produce a uniform distribution over the next token, getting terrible performances)

Answer 2

If your data contains a lot of near repeated samples, it’s quite possible too many training data can hurt, assuming you are training for 1 epoch. This is the same phenomenon as overfitting when you train on the same dataset multiple epochs.

Another reason to see degradation is if your increased training data is of lower quality.