Questions tagged [seq2seq]
For questions related to sequence-to-sequence (seq2seq) machine learning models/architectures, used e.g. in machine translation.
14
questions
1
vote
2answers
18 views
Is seq2seq the best model when input/output sequences have fixed length?
I understand that seq2seq models are perfectly suitable when the input and/or the output have variable lengths. However, if we know exactly the input/output sequence lengths of the neural network. Is ...
1
vote
0answers
42 views
What is the best way to generate German paraphrases?
What is the best method to generate German paraphrases? The state-of-the-art are seq2seq transformer models, like T5, but they only work for English sentences. I found the multilingual MT5 model, but ...
0
votes
0answers
18 views
Good metrics and losses to use for Sequence-to-Sequence model for time-series prediction/forecasting
I am developing a sequence-to-sequence LSTM model for multi-step time series forecasting. I have the basic model working, so now I need to drill down on which loss function and evaluation metrics to ...
3
votes
1answer
52 views
Can Reinforcement Learning be used to generate sequences?
Can we use reinforcement learning for sequence-to-sequence tasks? If yes, whether or not this is a good choice, how could this be done?
1
vote
0answers
20 views
Are there any successful applications of transformers of small size (<10k weights)?
In the problems of NLP and sequence modeling Transformer architectures based on self-attention mechanism https://arxiv.org/abs/1706.03762 have achieved impressive results and now are the first choices ...
1
vote
0answers
14 views
Training seq2seq translation model with one source and multiple target
So basically I'm training a sequence to sequence model that translates English sentences to Arabic sentences. I'm using the data provided by Anki @ manythings. I realized that some of the sentences in ...
0
votes
0answers
42 views
What is the role of this initial state of the LSTM layer in an encoder of a seq2seq model?
I am trying to follow this guide to implement a seq2seq machine translation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention.
The tutorial's ...
0
votes
1answer
112 views
Seq2Seq model produces repeating words
My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
3
votes
0answers
47 views
What is the difference between zero-padding and character-padding in Recurrent Neural Networks?
For RNN's to work efficiently, we vectorize the operations, which results in an input matrix of shape
(m, max_seq_len)
where m ...
2
votes
1answer
193 views
What's the difference between content-based attention and dot-product attention?
I'm following this blog post which enumerates the various types of attention.
It mentions content-based attention where the alignment scoring function for the $j$th encoder hidden state with respect ...
1
vote
1answer
305 views
How is Google Translate able to convert texts of different lengths?
According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
1
vote
0answers
102 views
SeqGAN - Policy gradient objective function interpretation
Could someone clear my doubt on the loss function used in SeqGAN paper . The paper uses policy gradient method to train the generator which is a recurrent neural network here.
Have I interpreted the ...
3
votes
0answers
480 views
What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?
I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
5
votes
2answers
6k views
What exactly is a hidden state in an LSTM and RNN?
I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its hidden states. In my specific case, the hidden state of ...
