All Questions
1 question
2
votes
0
answers
697
views
Where should we place layer normalization in a transformer model?
In Attention Is All You Need paper:
That is, the output of each sub-layer is $LayerNorm(x+Sublayer(x))$, where $Sublayer(x)$ is the function implemented by the sub-layer itself. We apply dropout to ...
- 121