Recently I came across the paper that introduces the Vision Transformer (ViT) "AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE".
The thing I don't really understand at the moment is, what is meant with "mean attention distance".
More specifically in the caption of Figure 11 on page 18 of the paper they state:
" ... Attention distance was computed for 128 example images by averaging the distance between the query pixel and all other pixels, weighted by the attention weight. ..."
How can the query be on a pixel level?
Isn't the overall approach of the ViT to divide the input image into patches which are linearly embedded, combined with a positional embedding and then feed into the transformer encoder.
So the attention should be on the patch level not on the pixel level?
I would be very happy if someone could elaborate a bit more on the above sentence, so far I found no deeper explanation.