I am really trying to understand deep learning models like RNN, LSTMs etc. I have gone through many tutorials of RNN and have learned that RNN cannot work for long Range dependencies, like:
Consider trying to predict the last word in the text “I grew up in France… I speak fluent French.” Recent information suggests that the next word is probably the name of a language, but if we want to narrow down which language, we need the context of France, from further back. It’s entirely possible for the gap between the relevant information and the point where it is needed to become very large. Unfortunately, as that gap grows, RNNs become unable to learn to connect the information.
it comes due to vanish gradient problem. However, I could not understand that how to vanish gradient creates an issue for RNN to not work for long-range dependencies. Since, as I know that vanish gradient usually comes when we have many hidden layers and the gradient for the first layer usually produced too low and that affects the training process. However, everyone connects this issue with vanish gradient, technically what is the relationship RNN (long-range dependencies) with vanish gradient?
I am really sorry if it is a weird question