3
$\begingroup$

While we train a CNN model we often experiment with the number of filters, the number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial & error process.

  1. Is there a way that helps me to architect my model fundamentally before training?

  2. Once I train model, how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?

P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.

Improve this question
$\endgroup$
3
  • $\begingroup$ Don't forget the possibility that the relationship between the data and target is not strong in the first place. CNNs are constrained by reality. If you have a really great labeled dataset of cat pictures and traits of their owner, it is fairly unlikely you could perform better than statistical average of labels when tryin to infer e.g. the cat's owner's shoe size from a picture of the cat. $\endgroup$
    – Neil Slater
    Sep 23, 2018 at 21:26
  • $\begingroup$ @NeilSlater you are right but in my case I have ruled out the possibility of non architectural flaw for the ease of the answer. However you are very right :) $\endgroup$
    – Ruchit Dalwadi
    Sep 24, 2018 at 21:31
  • $\begingroup$ I don't know why no one has pointed this out but you can get many clues by seeing loss function of train and CV set....Which can give you very good pointers about over or undercutting, learning rate and even optimisation methods maybe. $\endgroup$
    – user9947
    Oct 16, 2018 at 17:05

1 Answer 1

Reset to default
2
$\begingroup$

See Analysis and Optimization of Convolutional Neural Network Architectures (my master's thesis).

  1. Is there a way that helps me to architect my model fundamentally before training?

Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. https://autokeras.com/

See Chapter 3.

  1. Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?

See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.

Improve this answer
$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .