Questions tagged [seq2seq]
For questions related to sequence-to-sequence (seq2seq) machine learning models/architectures, used e.g. in machine translation.
16
questions
1
vote
1answer
18 views
Does Seq2Seq decoder take a special vector or the weights of the last encoder cell as an output?
I'm reading Sequence to Sequence Learning with Neural Networks and there's a thing that I couldn't quite grasp.
Paper says the encoder outputs a vector to be fed to the decoder. More precisely
Our ...
1
vote
2answers
20 views
Is seq2seq the best model when input/output sequences have fixed length?
I understand that seq2seq models are perfectly suitable when the input and/or the output have variable lengths. However, if we know exactly the input/output sequence lengths of the neural network. Is ...
1
vote
0answers
61 views
What is the best way to generate German paraphrases?
What is the best method to generate German paraphrases? The state-of-the-art are seq2seq transformer models, like T5, but they only work for English sentences. I found the multilingual MT5 model, but ...
0
votes
0answers
19 views
Good metrics and losses to use for Sequence-to-Sequence model for time-series prediction/forecasting
I am developing a sequence-to-sequence LSTM model for multi-step time series forecasting. I have the basic model working, so now I need to drill down on which loss function and evaluation metrics to ...
3
votes
1answer
53 views
Can Reinforcement Learning be used to generate sequences?
Can we use reinforcement learning for sequence-to-sequence tasks? If yes, whether or not this is a good choice, how could this be done?
1
vote
0answers
21 views
Are there any successful applications of transformers of small size (<10k weights)?
In the problems of NLP and sequence modeling Transformer architectures based on self-attention mechanism https://arxiv.org/abs/1706.03762 have achieved impressive results and now are the first choices ...
1
vote
0answers
18 views
Training seq2seq translation model with one source and multiple target
So basically I'm training a sequence to sequence model that translates English sentences to Arabic sentences. I'm using the data provided by Anki @ manythings. I realized that some of the sentences in ...
0
votes
0answers
46 views
What is the role of this initial state of the LSTM layer in an encoder of a seq2seq model?
I am trying to follow this guide to implement a seq2seq machine translation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention.
The tutorial's ...
0
votes
1answer
165 views
Seq2Seq model produces repeating words
My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
3
votes
0answers
50 views
What is the difference between zero-padding and character-padding in Recurrent Neural Networks?
For RNN's to work efficiently, we vectorize the operations, which results in an input matrix of shape
(m, max_seq_len)
where m ...
2
votes
1answer
301 views
What's the difference between content-based attention and dot-product attention?
I'm following this blog post which enumerates the various types of attention.
It mentions content-based attention where the alignment scoring function for the $j$th encoder hidden state with respect ...
2
votes
1answer
308 views
How is Google Translate able to convert texts of different lengths?
According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
1
vote
0answers
106 views
SeqGAN - Policy gradient objective function interpretation
Could someone clear my doubt on the loss function used in SeqGAN paper . The paper uses policy gradient method to train the generator which is a recurrent neural network here.
Have I interpreted the ...
3
votes
0answers
541 views
What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?
I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
1
vote
1answer
456 views
Do Seq2Seq models and the Bidirectional RNN do the same thing?
It seems to me that Seq2Seq models and Bidirectional RNNs try to do the same thing. Is that true?
Also, when would you recommend one setup over another?
5
votes
2answers
7k views
What exactly is a hidden state in an LSTM and RNN?
I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its hidden states. In my specific case, the hidden state of ...
