Collecting and labeling training data for supervised learning tasks is incredibly time-consuming and costly.
For instance, let's say you wrote a script that went on Google images and got you 5000 pictures for each of 10 classes. You then use an unsupervised algorithm to cluster them. Then, you train a supervised algorithm using the labels from the scraper as ground truth. Obviously, your network will perform more poorly than one with perfectly labeled data, but is there a way to guesstimate how much?
Perhaps there are 50 mislabeled images in each class. That would most likely be better than 500 mislabeled images, but I'm wondering if there is a way to predict how much (even if it is by someone's rules of thumb or something like that).