If I am attempting to train a CNN on some image data to perform image classification, but some of the images have pieces of text overlaying them (for the purpose of description to humans), then is it better for the CNN to remove the text? And if so, then how do I remove the text? Furthermore, is it a good idea to use both the images with text overlaying them and the images with the removed text for training, since it might act as a form of data augmentation?
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Removing the overlayed text might increase accuracy, but you'd need to train a different model to do this, and that is an entirely different task as it is no longer classification, but generation. There are easier ways to augment your data and probably get similar benefits to your accuracy. However, if you would still like to do this, there is a lot of examples you can find by simple searching "Watermark removal machine learning" in google. Here's an example I found.
Overall, a CNN will be able to look past the overlayed text without issue, and perform classification like it would without the overlayed text. There is the possibility that it actually learns relationships between the overlayed text and the expected output, but that depends on the data, and is likely a harder task then simply identifying features.
The only issue you might run into is if the real data this model will be used on is different to the data provided, as in the real word images do not contain overlayed text describing what the image is.
-
$\begingroup$ 1. I should have said: Not all of the data will have text overlaying it, so I guess that would help train the CNN to look past the text as "noise"? $\endgroup$ Sep 18, 2020 at 5:05
-
$\begingroup$ 2. Furthermore, with regards to my other question, if I use some method to remove the overlaying text, then should I still keep both versions of the images for training, just treating it the same way as we would augmented data (for the same benefit as we strive for when augmenting data)? Or is there some downside scenario where keeping the images with overlaying text actually decreases performance somehow? $\endgroup$ Sep 18, 2020 at 5:06
-
1$\begingroup$ 1. The CNN will probably then just learn to ignore the text, but it depends on how many images contain the text and how many don't. 2. No if the testing dataset contains images with text you should definitely keep these images in the training data. The only scenario I'd say removing the text with some method is necessary is if the amount of images without text is very small when compared to the amount that does have text. That way you will have a better balanced dataset, and the network shouldn't under-perform on images without text $\endgroup$ Sep 18, 2020 at 5:13