Why final memory state equals to the last hidden state of entire hidden state sequence?

Question

when return_sequences=True and return_state=True, a TensorFlow LSTM outputs the hidden states of the LSTM cell along with the memory state and hidden state of the cell as described in Tensorflow docs

lstm = tf.keras.layers.LSTM(4, return_sequences=True, return_state=True)
whole_seq_output, final_memory_state, final_carry_state = lstm(inputs)

So in my understanding, the last timestep of each item of whole_seq_output should be the same as final_carry_state, but it turned out it's not. Could someone please explain to me why?

whole_seq_output[:, -1, :] - final_carry_state # this should be equals to 0, but its not
whole_seq_output[:, -1, :] - final_memory_state # this equals to 0

Answer 1

This image shows how an lstm cell operates (taken from Wikipedia). The carry state as implemented in tensorflow is the c_t which is different from the final memory state h_t (which is also the state passed as sequence output). So seq_output - h_t = 0 and seq_output - c_t != 0.