# Is there an efficient way of determining the layers with the best performance as feature extractors in GoogleNet?

I am using a caffe model of pre-trained GoogleNet trained on ImageNet from here for image retrieval task (place recognition, more specifically).

I would like to know the layer with best performance in feature extraction. Its official paper suggests that:

The strong performance of shallower networks on this task suggests that the features produced by the layers in the middle of the network should be very discriminative.

There is also a project deepdream suggests that:

The complexity of the details generated depends on which layer's activations we try to maximize. Higher layers produce complex features, while lower ones enhance edges and textures, giving the image an impressionist feeling.

Searching the web, I found a github page suggesting pool5/7x7_s1layer as feature extractor without specific convincing reasons.

What I am doing now is quite cumbersome in which I extract features from each individual layer, apply scipy euclidean distance measurement to find a query in the reference database and the judgment is based on precision-recall curve and my top 3 results are as follows for one dataset:

1. inception_3a/3x3
2. inception_4a/5x5
3. inception_4b/output

Considering large number of convolutional layers in GoogleNet, my approach is undoubtedly quite inefficient and can be changed to another dataset!

Can anyone suggest an efficient way to figure out the layers with the best performance as feature extractors in GoogleNet?

• So, given a trained model, are you trying to understanding which of the layers of the pre-trained network extract the most "relevant" features? – nbro Mar 14 at 21:19
• That’s exactly what I’m looking for in googlenet or if there’s any rule of thumb for all networks – Färid Alijani Mar 14 at 21:26
• Maybe you're looking for something like "layer-wise relevance propagation", which is an explainable AI technique. – nbro Mar 14 at 21:27
• not LRP specifically! but more of using specific layers of GoogleNet as holistic feature extraction for problems other than image classification or object detection! What I work is, for example, place recognition problem in which I investigate layers of GoogleNet to figure out ones is more efficient finding more query images in reference database! – Färid Alijani Mar 24 at 13:27