I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. Was there a study on this? Or are there any past experiences anyone could share to help?

  • 1
    $\begingroup$ Intuitively, to minimize the biases introduced by the drawings you may want to have your drawings equally distributed among all the different classes that you have. Otherwise the system is likely to focus on the particular traits that constitute a drawing to predict the classes. Maybe applying some kind of style transfer as a preprocessing step to render all your input data alike could be an interesting idea to explore. $\endgroup$ – Jeanba Jan 10 at 14:16
  • 1
    $\begingroup$ I think if the drawings are realistic drawings they maybe used for weight pretraining before actual training if the data size is large enough. If the drawings are not realistic (for example maybe some impressionism drawings) then it should not be used. I don't think the drawings should be trained along side the real images as it may disrupt the accuracy and training process $\endgroup$ – Clement Hui Jan 11 at 4:46
  • $\begingroup$ @ClementHui When you say weight pretraining, do you mean that I should unfreeze all layers?And by "data size large enough" how much is large enough? $\endgroup$ – user Jan 11 at 8:11
  • $\begingroup$ When doing image detection, you should try using one of the available models like ResNet with imageNet weights. Then try training the imageNet model on the drawings but only unfreeze the last few layers. Then you should train on the main dataset. $\endgroup$ – Clement Hui Jan 11 at 8:17
  • $\begingroup$ By large enough I mean like 1000/class or more $\endgroup$ – Clement Hui Jan 11 at 8:18

To my knowledge the deployment model (that you will test on underwater images) as inference will not have a negative effect. Yet drawings may even help differentiate some classes at training and inference. Provided that you won't use drawings in inference, adding them in training phase will not necessarily hurt the accuracy. Note that a drawing of a particular class should not be in the search domains of other classes, namely, a drawing of a particular class should not be the same with the other classes.

| improve this answer | |
  • $\begingroup$ Though if I were to do it I would use the drawings for pretraining the weights, if the drawings data is big enough and if the drawings are about underwater images as well. $\endgroup$ – Clement Hui Jan 10 at 2:09
  • $\begingroup$ @voo_doo I don't quite understand this bit: " a drawing of a particular class should not be the same with the other classes". $\endgroup$ – user Jan 10 at 7:50
  • $\begingroup$ @bookfreak the author is saying that there are drawings of a certain class (say, a particular sea-grass). That drawing should not be similar in look to other real underwater images of other classes. If that makes sense... $\endgroup$ – bit_scientist Jan 10 at 9:26
  • $\begingroup$ @voo_doo Do you have a link? It would be better to read the whole thing. $\endgroup$ – user Jan 10 at 9:35
  • $\begingroup$ @bookfreak I guess you have the data set, post some of the drawings and other underwater images of other classes here as an edit. $\endgroup$ – bit_scientist Jan 10 at 9:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.