For questions related to the exploding gradient problem, which is the numerical problem associated with the significant increase (or explosion) of the numbers of the gradient vector of an objective function with respect to the parameters of a neural network, which is being trained with a gradient-based optimization algorithm and backpropagation. There is also the related vanishing gradient problem, which arises when the numbers become very small.
Stats
|
created |
3 years, 8 months ago |
|
viewed |
4 times |
|
editors |
1 |
Top Answerers
more »
Recent Hot Answers
Why do ResNets avoid the vanishing gradient problem?How to deal with large (or NaN) neural network's weights?
What is the justification for this approach of clipping elementwise?
Mathematically speaking, Is it only the product operation used in the chain rule causing the vanishing or exploding gradient?
What are the common pitfalls that we could face when training neural networks?
more »