Problem extracting features from convolutional layer where the dimensions are big for feature maps

Question

I have trained a convolutional neural network on images to detect emotions. Now I need to use the same network to extract features from the images and use them to train an LSTM. The problem is: the dimensions of the top layers are: [None, 4, 4, 512] or [None, 4, 4, 1024]. Therefore, extracting features from this layer will result in a 4 x 4 x 512 = 8192 or 4 x 4 x 1024 = 16384 dimensional vector for each image. Clearly, this is not what I want.

Therefore, I would like to know what to do in this case and how to extract features that are of reasonable size. Should I apply global average pooling to the activation or what?

Any help is much appreciated!

Answer 1

I think it's okay to let CNN extract a big number of feature maps, then you can reduce the features from it. To reduce features from CNN result, you can do feature selection or feature extraction. Some people prefer to use feature selection and select the most significant feature maps, for examples:

• Action unit selective feature maps in deep networks for facial expression recognition
• Convolutional Neural Network Simplification Based on Feature Maps Selection

Hope it helps.

Answer 2

In addition to the first answer about feature selection, you could also add a global max or average pooling layer at the end of your network. This would reduce the dimensionality to 512 or 1024. If that's still too much, another option would be to add an additional convolutional layer with reduced channels and then do the global pooling. You will have to experiment with which option is best for your data.