Questions tagged [encoder-decoder]
The encoder-decoder tag has no usage guidance.
8
questions
0
votes
1
answer
44
views
What is a "mask" in the context o RNN-based encoders?
While reading source code related to RNN encoders, I've come across the term mask as input to the encoder. What exactly is it?
- 3,099
0
votes
0
answers
16
views
U-Net with freezed encoder and customize decoder
I want to use a U-Net architecture, but I want only to use the feature extraction part and train the decoder for my task, but I'm not sure if the decoder of U-Net would be enough for learning new ...
- 33
0
votes
1
answer
315
views
What is input (and shape) to K/V/Q of self-attention of EACH Decoder block of Language-translation model Transformer's tokens during Inference?
Transformer model of the original Attention paper has a decoder unit that works differently during Inference than Tranining.
I'm trying to understand the shapes used during decoder (both self-...
- 151
0
votes
0
answers
66
views
What would be the total number of learnable parameters of the RNN encoder of this encoder-decoder architecture for machine translation?
Here's a quiz. My answer is different from the teacher's, so I'm wondering what answer would you pick up.
We use a sequence-to-sequence (encoder-decoder) system to perform
machine translation. We ...
- 113
0
votes
1
answer
313
views
Seq2Seq model produces repeating words
My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
1
vote
1
answer
88
views
How is the transformers' output matrix size arrived at?
In this tensorflow article, the comments in the code say that MHA should output with one of the dimensions being the sequence length of the query/key. However, that means that the second MHA in the ...
- 111
8
votes
4
answers
11k
views
What exactly is a hidden state in an LSTM and RNN?
I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its hidden states. In my specific case, the hidden state of ...
- 597
3
votes
1
answer
1k
views
Why do we need both encoder and decoder in sequence to sequence prediction?
Why do we need both encoder and decoder in sequence to sequence prediction?
We could just have a single RNN that, given input $x$, outputs some value $y(t)$ and hidden state $h(t)$. Next, given $h(t)$...
- 51