Questions tagged [gradient-clipping]
The gradient-clipping tag has no usage guidance.
2
questions
0
votes
0
answers
75
views
What is the effect of gradient clipping by norm on the performance of a model?
It is recommended to apply gradient clipping by normalization in case of exploding gradients. The following quote is taken from here answer
One way to assure it is exploding gradients is if the loss ...
- 2,975
2
votes
1
answer
222
views
What exactly happens in gradient clipping by norm?
Consider the following description regarding gradient clipping in PyTorch
...
- 2,975