Hot answers tagged

3

Usually, in natural language processing (NLP), they are using Sequence to Sequence Learning (Seq2Seq) with Neural Networks, such as Recurrent Neural Networks or more recently the Transformer (you can find two very good papers here, and here). During training, to ensure the same size of the input and output they can just search for the longest sentence they ...


3

This is my own understanding of hidden state in a recurrent network and if its wrong please feel free to let me know. Lets take this simple sequence first, X = [a,b,c,d,.......,y,z] Y = [b,c,d,e,.......,z,a] Instead of RNN we will first try to train this in a simple multi layer neural network with one input and one output, here hidden layers details ...


2

As you said, one way to look at it is definitely that the LSTM-encoder's encoding can be only understood by itself, that's why the decoder exists there. An optimisation process encoded it, why couldn't an optimisation process decode it? The hidden state is essentially just an encoding of the information you gave it keeping the time-dependencies in check. ...


Only top voted, non community-wiki answers of a minimum length are eligible