0
$\begingroup$

The Transformer model proposed in "Attention Is All You Need" uses sinusoid functions to do positional encoding. I wonder why do both sin and cos been involved in? And why do need to separate the odd and even dimentions to use differnent sinusoid functions?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.