I am wondering what is believed to be the reason for superiority of transformer?
I see that some people believe because of the attention mechanism used, it’s able to capture much longer dependencies. However, as far as I know, you can use attention also with RNN architectures as in the famous paper attention is introduced(here)).
I am wondering whether the only reason for the superiority of transformers is because they can be highly parallelized and trained on much more data?
Is there any experiment comparing transformers and RNN+attention trained on the exact same amount of data comparing the two?