Is a VGG-based CNN model sometimes better for image classfication than a modern architecture?

Question

I have an image classification task to solve, but based on quite simple/good terms:

• There are only two classes (either good or not good)
• The images always show the same kind of piece (either with or w/o fault)
• That piece is always filmed from the same angle & distance
• I have at least 1000 sample images for both classes

So I thought it should be easy to come up with a good CNN solution - and it was. I created a VGG16-based model with a custom classifier (Keras/TF). Via transfer learning I was able to achieve up to 100% validation accuracy during model training, so all is fine on that end.

Out of curiosity and because the VGG-based approach seems a bit "slow", I also wanted to try it with a more modern model architecture as the base, so I did with ResNet50v2 and Xception. I trained both similar to the VGG-based model, tried it with several hyperparameter modifications, etc. However, I was not able to achieve a better validation accuracy than 95% - so much worse than with the "old" VGG architecture.

Hence my question is:

Given these "simple" (always the same) images and only two classes, is the VGG model probably a better base than a modern network like ResNet or Xception? Or is it more likely that I messed something up with my model or simply got the training/hyperparameters not right?

Answer 1

VGG is a more basic architecture which uses no residual blocks. Reset usually perform better then VGG due to it's more layers and residual approach. Given that resnet-50 can get 99% accuracy on MNIST and 98.7% accuracy on CIFAR-10, it probably should achieve better than VGG network. Also, the validation accuracy should not be 100%. You could try increasing the size of your validation set to improve accuracy on validation. VGG network should perform worst than ResNet in most scenario, but experimenting is the way to go. Try and experiment more to get a method that works for your data. Hope that I can help you and have a nice day!

Answer 2

Below is a listing of Keras application models that can be used easily in transfer learning. Note VGG has on the order of 140 million parameters which is why it is slow.

Model               Size     Top-1 Accuracy  Top-5 Accuracy  Parameters    1Depth
Xception             88 MB      0.790           0.945         22,910,480    126
VGG16               528 MB      0.713           0.901        138,357,544    23
VGG19               549 MB      0.713           0.900        143,667,240    26
ResNet50             98 MB      0.749           0.921         25,636,712    -
ResNet101           171 MB      0.764           0.928         44,707,176    -
ResNet152           232 MB      0.766           0.931         60,419,944    -
ResNet50V2           98 MB      0.760           0.930         25,613,800    -
ResNet101V2         171 MB      0.772           0.938         44,675,560    -
ResNet152V2         232 MB      0.780           0.942         60,380,648    -
InceptionV3          92 MB      0.779           0.937         23,851,784    159
InceptionResNetV2   215 MB      0.803           0.953         55,873,736    572
MobileNet            16 MB      0.704           0.895          4,253,864    88
MobileNetV2          14 MB      0.713           0.901          3,538,984    88
DenseNet121          33 MB      0.750           0.923          8,062,504    121
DenseNet169          57 MB      0.762           0.932         14,307,880    169
DenseNet201          80 MB      0.773           0.936         20,242,984    201
NASNetMobile         23 MB      0.744           0.919          5,326,716    -
NASNetLarge         343 MB      0.825           0.960         88,949,818    -

I tend to use the MobileNet model for transfer learning because it has about 4 million
parameters so it much faster than most models. It should perform as well as VGG on your
data set. If it does not tuning the hyper parameters may be required. I find that using
an adjustable learning such as the Keras ReduceLROnPlateau callback along with the
ModelCheckpoint callback both monitoring validation loss works very well. Documentation
is [here][1].
You might also try the efficientNet model which comes in various sizes and has high 
accuracy. Documentation is [here][2]


  [1]: https://keras.io/callbacks/
  [2]: https://github.com/Tony607/efficientnet_keras_transfer_learning

Answer 3

The newer models generally outperform older ones on the ImageNet challenge in their accuracy scores*. This does not necessarily mean that this difference in performance will be reflected in your particular classification problem.

The closer your problem is to the ImageNet one, the more likely that the relative model performances will be similar. However when you perform transfer learning you will often have to fine-tune the model to achieve a stronger performance, the better you tune the model will effect performance, and there will often be a difference in which model is performing best on a given task. You can see papers in various classification tasks where VGG may be performing best, or Inception, or even AlexNet. I believe the simplest models (AlexNet has only 8 layers) may be the easiest to fine tune, and also may require the smallest amount of data for good performance.

*There are exceptions, MobileNet is more recent but the innovation is that it is a smaller model rather than the strongest model i.e. it is designed to be useable on mobile devices rather than running on the latest GPU.