Questions tagged [seq2seq]
For questions related to sequence-to-sequence (seq2seq) machine learning models/architectures, used e.g. in machine translation.
5
questions
3
votes
0answers
29 views
What's the difference between content-based attention and dot-product attention?
I'm following this blog post which enumerates the various types of attention.
It mentions content-based attention where the alignment scoring function for the $j$th encoder hidden state with respect ...
1
vote
1answer
279 views
How is Google Translate able to convert texts of different lengths?
According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
1
vote
0answers
87 views
SeqGAN - Policy gradient objective function interpretation
Could someone clear my doubt on the loss function used in SeqGAN paper . The paper uses policy gradient method to train the generator which is a recurrent neural network here.
Have I interpreted the ...
2
votes
0answers
194 views
What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?
I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
3
votes
2answers
2k views
What exactly is a hidden state in an LSTM and RNN?
I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its hidden states. In my specific case, the hidden state of ...
