I have achieved around 85% accuracy using the following architecture:
I used a learning rate of 0.001 and trained the model over 125 epochs with a batch size of 64. Any suggestions would be much appreciated. Thanks in advance.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Try applying a grid search to all your hyper parameters (such as dropout and learning rate). A grid search is easy to do and can improve your accuracy a lot. If you really wanted to know the best architecture, search up NAS (Neural Architecture Search). Keep in mind this is significantly more complicated. $\endgroup$– RecessiveNov 19, 2019 at 5:50
-
1$\begingroup$ Frankly 85% is way too good for the architecture you are using, I couldn't achieve 65% percentage with a similar architecture (just a little smaller) $\endgroup$– user9947Nov 21, 2019 at 7:52
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You results show signs of overfitting at around epoch 40. In order to overcome this you can either simplify the model somewhat or increase regularization. You do not share what values you are using for dropout regularization so you can try increasing that.
But to be honest, I am not sure if that is going to help. You are using dropout in a pure CNN architecture, which I do not see that often in recent models. When they exists, they are usually at the very end of the network, or after densely connected layers. There are claims that regular dropout does not work as intended for convolutional layers, so the idea of spatial dropout is developed.
But you are using them after pooling layers and such a design may not be as bad as using them after convolutional layers. In any case, if you check the top performers on this dataset, you can see there are some who are trying to come up with new regularization techniques for CNN's, such as shake-shake, shakedrop and cutout.
However, given you already have an overfitting model, removing regularization might not be the best idea. But if you replace the layers after (and including) the last convolution with some fully connected layers, you can use dropout regularization on them instead. This would remove the largest convolutional layer of the model and you can add dense layers without increasing model size.
For image recognition tasks of any kind, easiest way to achieve high accuracy is via transfer learning.
