Skip to main content

Questions tagged [normalisation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
596 views

In the attention mechanism, why don't we normalize after multiplying values?

As this question says: In scaled dot product attention, we scale our outputs by dividing the dot product by the square root of the dimensionality of the matrix: The reason why is stated that this ...
Peyman's user avatar
  • 574