Questions tagged [normalisation]
The normalisation tag has no usage guidance.
1
question
0
votes
1
answer
150
views
In the attention mechanism, why don't we normalize after multiplying values?
As this question says:
In scaled dot product attention, we scale our outputs by dividing the
dot product by the square root of the dimensionality of the matrix:
The reason why is stated that this ...
- 454