Questions tagged [seq2seq]

For questions related to sequence-to-sequence (seq2seq) machine learning models/architectures, used e.g. in machine translation.

Filter by
Sorted by
Tagged with
2
votes
0answers
34 views

What is the difference between zero-padding and character-padding in Recurrent Neural Networks?

For RNN's to work efficiently, we vectorize the operations, which results in an input matrix of shape (m, max_seq_len) where m ...
2
votes
0answers
37 views

What's the difference between content-based attention and dot-product attention?

I'm following this blog post which enumerates the various types of attention. It mentions content-based attention where the alignment scoring function for the $j$th encoder hidden state with respect ...
1
vote
1answer
289 views

How is Google Translate able to convert texts of different lengths?

According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
1
vote
0answers
92 views

SeqGAN - Policy gradient objective function interpretation

Could someone clear my doubt on the loss function used in SeqGAN paper . The paper uses policy gradient method to train the generator which is a recurrent neural network here. Have I interpreted the ...
3
votes
0answers
326 views

What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?

I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
4
votes
2answers
4k views

What exactly is a hidden state in an LSTM and RNN?

I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its hidden states. In my specific case, the hidden state of ...