I am trying to follow this guide to implement a seq2seq machine translation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention.

The tutorial's Encoder has an initialize_hidden_state() function that is used to generate all 0 as initial state for the encoder. However, I am a bit confused as to why this is necessary. As far as I can tell, the only times when encoder is called (in train_step and evaluate), they were initialized with the initialize_hidden_state() function.

My questions are

  1. What is the purpose of this initial state? Doesn't the Keras layer automatically initialize LSTM states, to begin with?

  2. Why not always just initialize the encoder with all 0 hidden states if encoder is always called with initial states generated by initialize_hidden_state()?

  • $\begingroup$ initialize_hidden_state does not appear in this tutorial? $\endgroup$
    – user253751
    Aug 11 at 9:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.