1
$\begingroup$

I came across the concept of "deep learning primitives" from the Nvidia talk Jetson AGX Xavier New Era Autonomous Machines (on slide 44).

There doesn't seem to be a lot of articles in the community on this concept. I was able to find one definition from here, where it defined deep learning primitives as the "fundamental building blocks of deep networks" like fully connected layers, convolutions layers, etc.

I was curious to find out if a self-attention layer is a primitive, I came across this OpenDNN issue and one person explained that self-attention layers can be built by other primitives like inner product, concat, etc.

So my question is what exactly are primitives in deep learning? What makes a convolution layer a primitive and a self-attention layer not a primitive?

Improve this question
$\endgroup$
2
  • $\begingroup$ Slide 44 only says "DEEP VISION PRIMITIVES" and then they provide examples of computer vision tasks, so I am not sure where, in that slide, you saw the term "deep learning primitives". $\endgroup$ – nbro♦ Nov 5 '20 at 15:59
  • $\begingroup$ I thought the concept of primitives would generalise to beyond just vision tasks? The second link gives a definition of deep learning primitives: oreilly.com/library/view/tensorflow-for-deep/9781491980446/… $\endgroup$ – Frank Nov 5 '20 at 17:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.