Skip to main content

All Questions

Filter by
Sorted by
Tagged with
0 votes
2 answers
2k views

Layer Norm in a ResNet MLP

Where do you insert layer norm in a residual block? After the addition or before the activation function (RELU in this case)?
postnubilaphoebus's user avatar