Questions tagged [encoder-decoder]
The encoder-decoder tag has no usage guidance.
6
questions
0
votes
0answers
9 views
Encode states based on after-the-fact Similarity?
The question is what can you do in case where encoding STATE SIMILARITY is dependent on the interaction i.e. it is known after the fact.
For example in chess you may have a State’s (S1 and S2) where a ...
0
votes
1answer
137 views
Seq2Seq model produces repeating words
My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
0
votes
0answers
16 views
Can an Autoencoder be used to create a simple text-completion plugin (code-completion for programming)?
Can an Autoencoder neural network be used to create a simple code completion plugin for a developer using a certain programming language ?
The idea is that the training data will be generated from ...
1
vote
1answer
62 views
How is the transformers' output matrix size arrived at?
In this tensorflow article, the comments in the code say that MHA should output with one of the dimensions being the sequence length of the query/key. However, that means that the second MHA in the ...
5
votes
2answers
7k views
What exactly is a hidden state in an LSTM and RNN?
I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its hidden states. In my specific case, the hidden state of ...
3
votes
2answers
860 views
Why do we need both encoder and decoder in sequence to sequence prediction?
Why do we need both encoder and decoder in sequence to sequence prediction?
We could just have a single RNN that, given input $x$, outputs some value $y(t)$ and hidden state $h(t)$. Next, given $h(t)$...
