Questions tagged [seq2seq]

For questions related to sequence-to-sequence (seq2seq) machine learning models/architectures, used e.g. in machine translation.

Filter by
Sorted by
Tagged with
5
votes
2answers
7k views

What exactly is a hidden state in an LSTM and RNN?

I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its hidden states. In my specific case, the hidden state of ...
0
votes
0answers
6 views

Could not compute output KerasTensor in multi modal seq2seq [migrated]

I am trying to generate comments from subreddit posts using images, titles, and source subreddit of the post. if you don't know what a subreddit is, just think about it as a category of the post e.g ...
1
vote
2answers
18 views

Is seq2seq the best model when input/output sequences have fixed length?

I understand that seq2seq models are perfectly suitable when the input and/or the output have variable lengths. However, if we know exactly the input/output sequence lengths of the neural network. Is ...
0
votes
0answers
51 views

What is the best way to generate German paraphrases?

What is the best method to generate German paraphrases? The state-of-the-art are seq2seq transformer models, like T5, but they only work for English sentences. I found the multilingual MT5 model, but ...
0
votes
0answers
19 views

Good metrics and losses to use for Sequence-to-Sequence model for time-series prediction/forecasting

I am developing a sequence-to-sequence LSTM model for multi-step time series forecasting. I have the basic model working, so now I need to drill down on which loss function and evaluation metrics to ...
3
votes
1answer
53 views

Can Reinforcement Learning be used to generate sequences?

Can we use reinforcement learning for sequence-to-sequence tasks? If yes, whether or not this is a good choice, how could this be done?
1
vote
0answers
21 views

Are there any successful applications of transformers of small size (<10k weights)?

In the problems of NLP and sequence modeling Transformer architectures based on self-attention mechanism https://arxiv.org/abs/1706.03762 have achieved impressive results and now are the first choices ...
1
vote
0answers
16 views

Training seq2seq translation model with one source and multiple target

So basically I'm training a sequence to sequence model that translates English sentences to Arabic sentences. I'm using the data provided by Anki @ manythings. I realized that some of the sentences in ...
0
votes
0answers
43 views

What is the role of this initial state of the LSTM layer in an encoder of a seq2seq model?

I am trying to follow this guide to implement a seq2seq machine translation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention. The tutorial's ...
0
votes
1answer
137 views

Seq2Seq model produces repeating words

My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
2
votes
1answer
223 views

What's the difference between content-based attention and dot-product attention?

I'm following this blog post which enumerates the various types of attention. It mentions content-based attention where the alignment scoring function for the $j$th encoder hidden state with respect ...
3
votes
0answers
50 views

What is the difference between zero-padding and character-padding in Recurrent Neural Networks?

For RNN's to work efficiently, we vectorize the operations, which results in an input matrix of shape (m, max_seq_len) where m ...
2
votes
1answer
308 views

How is Google Translate able to convert texts of different lengths?

According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
1
vote
0answers
105 views

SeqGAN - Policy gradient objective function interpretation

Could someone clear my doubt on the loss function used in SeqGAN paper . The paper uses policy gradient method to train the generator which is a recurrent neural network here. Have I interpreted the ...
3
votes
0answers
492 views

What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?

I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...