The Transformer model proposed in "Attention Is All You Need" uses sinusoid functions to do positional encoding. I wonder why do both sin and cos been involved in? And why do need to separate the odd and even dimentions to use differnent sinusoid functions?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.