New answers tagged recurrent-neural-networks
3
Yes, there are different ways. What I think you are looking for is under the research field of Localization and Mapping. Which divides in the following subfields:
For getting current (the robot) position and trajectory go to models for Odometry Estimation
For getting a representation of the world around the robot go to models for Mapping
If you want both of ...
1
Vanishing gradient is: as the gradient starts to flow from the end of the network (right side of the network) to the start of the network (left side of the network), it will be multiplied by numbers less than 1 and gradually it will become weaker and weaker and when it arrives to the first layers, it's so weak that makes almost no change in initial layers ...
1
Turns out the reason is because, for places where a dot is shown in the image above, they're actually element-wise multiplications, not dot products. A lot of sources use an X or . to denote multiplication, but don't clearly indicate they mean element-wise multiplication.
1
Hopfield networks, a special case of RNNs, were first proposed in 1982: https://www.pnas.org/content/79/8/2554
Otherwise (shameless plug, I am the author) a non-technical timeline for NLP can be found here: https://blog.exxcellent.de/ki-machine-learning
1
I have not implement the backprop of a bi-directional RNN from scratch so I can't be sure my answer is correct but I hope it helps.
You can see how bi-directional RNN works from this video from Andrew NG. I got the image below from that video:
For more clarity:
So if you know how to backprop through a simple RNN, you should be able to do so for bi-...
2
If you go through the main introductory paper of the transformer ("Attention is all you need"), you can find the comparison of the model with other state-of-the-art machine translation method:
For example, Deep-Att + PosUnk is a method that has utilized RNN and attention for the translation task. As you can see, the training cost for the ...
Top 50 recent answers are included
