I am trying to figure out the difference between the architecture used in this and this paper. It looks like both used multi-headed self-attention and therefore should be the same in principle.

  • 1
    $\begingroup$ I'm on top of the vision transformer but not the RL. But I know enough to tell you that these are very unrelated papers (within deep learning). And transformers are used across many types of tasks (vision, NLP, RL, audio). Imagine making an analogy from DL to all vehicles. In this context you can't exactly say a car is like a plane because they both have pistons. Same reason I wouldn't say these papers are particularly related by the fact that they both use transformers. $\endgroup$ Commented Aug 24, 2021 at 3:18

1 Answer 1


An Image is Worth 16X16 Words:


A Transformer consists of alternating layers of multiheaded self-attention.

The Transformer Paper adapts a NLP architecture for making Image Classification. For that, it first need to tokenize the image (like a piece of text). The tokenization is done by splitting the image into fixed-size patches and then embedding it.

In other words, they treat a picture as a set of sub-images, just like a phrase is a set of words.

Relational Deep Reinforcement Learning

The Reinforcement Learning paper uses self-attention to iteratively reason about the relations between entities in a scene and to guide a model-free policy.

Focused on game playing, they decide to model a non-local concept of entities (players, objects) and the relationship between them.

So instead of tokenizing the image, they first extract entities from it, using a more standard approach, which is convolution. So the multi-head attention layer receives a set of entities, the output is normalized so it's once again formatted as the entity set, so it can proceed with an iterative process.

TL;DR: Same base (very promising) technique, but applied to 2 drastically different models.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .