Image prediction model when data-set classes have visual similarity

Question

Lets say we have a data-set of all cats and we have to identify the cat breed based on given test image. As, the two different cat breeds have visual similarity can we use existing networks (VGG, ImageNet, GoogleNet) to solve this problem?

1. Should faceNet be applied here? As, the problem is similar to face detection where face characteristics of two different people are same yet it can correctly recognize a person.
2. What if with visual similarity in data-set we have only few example of each class? Like for a problem (random) we have good amount of data but for each class we have only few examples.

Is there any model that can be applied here?

Answer 1

You could use transfer learning. Find some state of the art neural net (something newer than VGG) already trained for image classification. Then remove the last layer and add a new one, training it while keeping the remaining old weights fixed. A simple softmax at the end would work.This is the simplest approach.

Because you're mostly only training the last layer, you won't need as much data.

I think FaceNet will perform poorly because it uses different features and you would need to retrain all its layers.

If you have too little data for some classes, maybe you should use an autoencoder as a features extractor. Then you could classify an image by comparing its feature vector to those for each breed, for example with cosine similarity.