2
$\begingroup$

Which part of this is the transformer?

enter image description here

Ok, the caption says the whole thing is the transformer, but that's back in 2017 when the paper was published. My question is about how the community uses the term "transformer" now.

I'm not looking for an inline response to these questions. They are all a way of asking the same general thing.

  • Is this whole thing a transformer?
  • What parts or what relationships between parts make it a transformer?
  • Equivalently, what aspects can I change before it becomes something else and not a transformer?
  • If I only care about self-attention I suppose I don't need the right hand column. If I just keep the self-attention, is it still a transformer?

Context about me is I've just become familiar with transformers and have not read much literature on them since this paper.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.