Questions tagged [encoder-decoder]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Encode states based on after-the-fact Similarity?

The question is what can you do in case where encoding STATE SIMILARITY is dependent on the interaction i.e. it is known after the fact. For example in chess you may have a State’s (S1 and S2) where a ...
0
votes
1answer
36 views

Seq2Seq model produces repeating words

My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
0
votes
0answers
15 views

Can an Autoencoder be used to create a simple text-completion plugin (code-completion for programming)?

Can an Autoencoder neural network be used to create a simple code completion plugin for a developer using a certain programming language ? The idea is that the training data will be generated from ...
1
vote
1answer
45 views

How is the transformers' output matrix size arrived at?

In this tensorflow article, the comments in the code say that MHA should output with one of the dimensions being the sequence length of the query/key. However, that means that the second MHA in the ...
4
votes
2answers
4k views

What exactly is a hidden state in an LSTM and RNN?

I'm working on a project, where we use an encoder-decoder architecture. We decided to use an LSTM for both the encoder and decoder due to its hidden states. In my specific case, the hidden state of ...