In the context of image classification, I am using a feature extractor based on a resnet-like architecture (ResNet12): four residual blocks, each of which is made of two consecutive conv3x3, batch norm and leaky relu activation function and a final conv3x3, batch norm, residual, leaky relu and maxpool. This architecture is mainly used in the context of few-shot learning, but for this case, consider a simpler image classification task.

I added an attention module (CBAM) to the main architecture. During training I plot a heatmap based on the attention score values retrieved and, over the first iterations, it seems like the feature extractor starts focusing on the correct regions of the image, which means that it should be able to identify the subject (some errors still occur, but consider that it is still the beginning of training).

The problem arises later when the attention "heatmap" starts covering the whole image, including the background and non-relevant parts. By debugging the values I noticed that the scores are all shrunk towards 1. Consider that the output of the CBAM layer is given by a sigmoid activation function, so the values are supposed to be in the [0-1] range (see the attached image).

In the first attempt, the attention module was embedded in every residual block, and then I tried to only use it in only one of them, but the results were conceptually the same. In addition, the attention module receives as input the values that come from the last batch normalization layer, at the end of the residual block.

Have you ever encountered such a problem? Do you have any clue on how could I solve it, or do you know any other attention mechanisms that could fit this feature extractor?

attention epoch 4 vs epoch 30



You must log in to answer this question.