1
$\begingroup$

While we train a CNN model we often experiment with number of filters, number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial & error process.

  1. Is there a way that helps me to architect my model fundamentally before training?

  2. Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?

P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.

$\endgroup$
  • $\begingroup$ Don't forget the possibility that the relationship between the data and target is not strong in the first place. CNNs are constrained by reality. If you have a really great labeled dataset of cat pictures and traits of their owner, it is fairly unlikely you could perform better than statistical average of labels when tryin to infer e.g. the cat's owner's shoe size from a picture of the cat. $\endgroup$ – Neil Slater Sep 23 '18 at 21:26
  • $\begingroup$ @NeilSlater you are right but in my case I have ruled out the possibility of non architectural flaw for the ease of the answer. However you are very right :) $\endgroup$ – Ruchit Dalwadi Sep 24 '18 at 21:31
  • $\begingroup$ I don't know why no one has pointed this out but you can get many clues by seeing loss function of train and CV set....Which can give you very good pointers about over or undercutting, learning rate and even optimisation methods maybe. $\endgroup$ – DuttaA Oct 16 '18 at 17:05
2
$\begingroup$

See Analysis and Optimization of Convolutional Neural Network Architectures.

  1. Is there a way that helps me to architect my model fundamentally before training?

Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. https://autokeras.com/

See Chapter 3.

  1. Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?

See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.

$\endgroup$
  • 1
    $\begingroup$ that research paper seems pretty useful. However I will dive deep into autokeras aswell. Thanks :) $\endgroup$ – Ruchit Dalwadi Sep 24 '18 at 21:30
  • $\begingroup$ I should probably mention that this is my masters thesis :-) $\endgroup$ – Martin Thoma Sep 25 '18 at 4:37
  • $\begingroup$ I figured out ;) $\endgroup$ – Ruchit Dalwadi Sep 25 '18 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.