2
$\begingroup$

I've found multiple depictions of how an LSTM cell operates. See 2 below:

enter image description here

and

enter image description here

Each of these images suggest the hidden state is utilised differently. On the top diagram, it is shown that the hidden state is added along with the previous output and current input to both the forget gate and the input gate. The bottom image suggests the input and forget gates are calculated only using the previous output and current input. Which is it?

Also, when the previous output is fed in for the current layer, is this before or after it has been reshaped to the final output size and been put through a softmax?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

  1. There are different variants of LSTM, in most ML packages now of days you'll probably see what's shown in the bottom picture. For more details, intuition and motivations please see this paper.

  2. It is not reshaped and there is no softmax layer. That is all done outside the LSTM.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .