0
$\begingroup$

I developed a CNN for image analysis. I've around 100K labeled images. I'm getting a accuracy around 85% and a validation accuracy around 82%, so it looks like the model generalize better than fitting. So, I'm playing with different hyper-parameters: number of filters, number of layers, number of neurons in the dense layers, etc.

For every test, I'm using all the training data, and it is very slow and time consuming.

Is there a way to have an early idea about if a model will perform better than another?

$\endgroup$
  • $\begingroup$ Hi, you don't specify the expected accuracy in your question. The difference that you have written between 85% for training and 82% for validation seems normal and implies that the model generalises well. What is your expected accuracy, and how do you know it is expected (is there another experiment using the exact same data that you are trying to match)? $\endgroup$ – Neil Slater Sep 6 '18 at 13:15
  • $\begingroup$ @NeilSlater yes, I think that it generalizes ok, even if I try to overfit I don't get better acc that 87% or so. Trained human accuracy could be around 98%. I've tried to use transfer learning with similar or worst results... The labeled data is made by my, no another experiment is using the exact data. $\endgroup$ – Mquinteiro Sep 6 '18 at 13:21
  • $\begingroup$ So your expectation of 98% is based on human accuracy on the same task? $\endgroup$ – Neil Slater Sep 7 '18 at 18:52
  • $\begingroup$ @NeilSlater yes it is, or something close to it, 95% could be enough, but I'm far, now playing manually with the number of convolution layers and filters. so tedious $\endgroup$ – Mquinteiro Sep 7 '18 at 22:42
0
$\begingroup$

I would like you to try with following changes in the model.

  • Introduce batch normalization layer in the model.
  • Try with the batch size of 32 - 64
  • Use different architecture, Like VGG, Resnet, etc..

There is no full proof answer to this question but you can get best by trying some know strategies.

$\endgroup$
  • $\begingroup$ Actually I did all your say 1) BN is the network. 2) The batch size is 50, tested with VGG19 and the results are similar to mine, also I have test with Vision AutoML from google with similar results. $\endgroup$ – Mquinteiro Sep 6 '18 at 14:47
0
$\begingroup$

Your description sounds like something similar to Imagenet dataset. According to this website, the state-of-the-art top-1 accuracy just as high as 86%, not much higher than yours. There are plenty of methods to improve accuracy. I would suggest you to read the paper or github listed in SOTA to find ideas that best fits your situation.

$\endgroup$
0
$\begingroup$

I can't comment but here are a few suggestions:

  • Play with lr and lr finder
  • Use a pretrained model
  • Use architecture search or use something like efficientnet b6
  • Use swish over relu
  • Try different optimizers
  • Try Bayesian Optimization
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.