# Attention mechanism: Why apply multiple different transformations to obtain query, key, value

I have two questions about the structure of attention modules:

Since I work with imagery I will be talking about using convolutions on feature maps in order to obtain attention maps.

1. If we have a set of feature maps with dimensions [B, C, H, W] (batch, channel, height, width), why do we transform our feature maps before we calculate their affinity/correlation in attention mechanisms? What makes this better than simply taking the cosine distance between the feature vectors (e.g. resizing the maps to [B, C, HW] and [B, HW, C] and multiplying them together). Aren't the feature maps already in an appropriate feature/embedding space that we can just use them directly instead of transforming them first?

2. Most of the time, attention mechanisms will take as input some stack of feature maps (F), and will apply 3 transformations on them to essentially produce a "query", "key" and "value". The query and key will be multiplied together to get the affinity/correlation between a given feature vector and all other feature vectors. In computer vision these transformation will typically be performed by the different 1x1 convolutions. My question is, how come we use 3 different 1x1 convolutions? Wouldn't it make more sense to apply the same 1x1 convolution to the input F? My intuition tells me that since we want to transform/project the feature maps F into some embedding/feature space that it would make the most sense if the "query", "key" and "value" were all obtained by using the same transformation. To illustrate what I mean lets pretend we had a 1x1 feature map and we wanted to see how well the pixel correlates with itself. Obviously it should correlate 100% because it is the same pixel. But wouldn't applying two sets of 1x1 convs to the pixel lead to the chance that the pixel would undergo a different transformation and in the end would have a lower correlation than it should?

• Hello. Welcome to AI SE. It seems to me that you're asking multiple questions here. When you have multiple questions, even if they are related, it's better to create multiple posts, one for each question (with the appropriate context to understand each post), so that people can focus on a single question at a time. Your question could be closed as "needs more focus" because you're asking many questions. Maybe the questions are very related. I didn't really/carefully read your post. See this.
– nbro
Commented Aug 3, 2021 at 9:53
• I'm also having some trouble understanding exactly what you are trying to ask. Can you possibly describe the model architectures you are asking about a bit more precisely? Commented Aug 3, 2021 at 14:27