5
$\begingroup$

It this podcast between Oriol Vinyals and Lex Friedman: https://youtu.be/Kedt2or9xlo?t=1769, at 29:29, Oriol Vinyals refers to a paper:

If you look at research in computer vision where it makes a lot of sense to treat images as two dimensional arrays... There is actually a very nice paper from Facebook. I forgot who the authors are but I think [it's] part of Kaiming He's group. And what they do is they take an image, which is a 2D signal, and they actually take pixel by pixel, and scramble the image, as if it was a just a list of pixels, crucially they encode the position of the pixels with the XY coordinates. And this is a new architecture which we incidentally also use in Starcraft 2 called the transformer, which is a very popular paper from last year which yielded very nice results in machine translation.

Do you know which paper he is referring to?

I'm guessing maybe he is talking about non-local neural networks, but I'm probably guessing wrong.

Edit: after reviewing the recent publications of Kaiming He (http://kaiminghe.com/), maybe I'm guessing right. Any thoughts?

Improve this question
$\endgroup$
2
  • $\begingroup$ If you verify please do update. I’d like to read that, too. Also, does this podcast have show notes? Sometimes they include links to referenced papers / blogs, etc. $\endgroup$
    – Hanzy
    May 1, 2019 at 16:47
  • 2
    $\begingroup$ This isnt the paper but it sounds related. "[An intriguing failing of convolutional neural networks and the CoordConv solution][1]" from Uber AI Labs. They also have a [really informative video][2] and a [repo][3]. [1]: arxiv.org/pdf/1807.03247.pdf [2]: youtube.com/watch?v=8yFQc6elePA [3]: github.com/uber-research/CoordConv $\endgroup$
    – Jaden Travnik
    May 1, 2019 at 17:18

1 Answer 1

Reset to default
2
$\begingroup$

So this paper is by google, but is very similar where they use 2D positional embeddings and perform MHA on the flattened image. Are you talking about Attention Augmented Convolutional Networks

Improve this answer
$\endgroup$
1
  • $\begingroup$ Looks like this is not "the" paper, but it's a good find, thanks $\endgroup$
    – Benjamin Crouzier
    May 13, 2019 at 19:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .