return_state=True, a TensorFlow LSTM outputs the hidden states of the LSTM cell along with the memory state and hidden state of the cell as described in Tensorflow docs
lstm = tf.keras.layers.LSTM(4, return_sequences=True, return_state=True) whole_seq_output, final_memory_state, final_carry_state = lstm(inputs)
So in my understanding, the last timestep of each item of
whole_seq_output should be the same as
final_carry_state, but it turned out it's not. Could someone please explain to me why?
whole_seq_output[:, -1, :] - final_carry_state # this should be equals to 0, but its not whole_seq_output[:, -1, :] - final_memory_state # this equals to 0