I am trying to follow this guide to implement a seq2seq machine translation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention.
Encoder has an
initialize_hidden_state() function that is used to generate all 0 as initial state for the encoder. However, I am a bit confused as to why this is necessary. As far as I can tell, the only times when
encoder is called (in
evaluate), they were initialized with the
My questions are
What is the purpose of this initial state? Doesn't the Keras layer automatically initialize LSTM states, to begin with?
Why not always just initialize the encoder with all 0 hidden states if
encoderis always called with initial states generated by