0
$\begingroup$

It does not come clear to me how the seq_length is not the exact same as the hidden_size in LSTMs.

For example, in the next image, the seq_length is T since there are $x_{1}, x_{2}, ..., x_{T}$ time steps and there are is one LSTM cell (green LSTM blocks) per input $x_{i}$. So in this image seq_length == hidden_size

In addition, every LSTM cell requires 3 inputs:

  1. $x_{t}$ the input value at time t.
  2. $h_{t-1}$ the hidden state value from the previous time step.
  3. $c_{t-1}$ the cell state value from the previous time step.

This can be seen in the following image:

How can the hidden_size be lower or higher than the seq_length? In any of both cases the LSTM cells would not have the $x_{i}$ input which is required.

$\endgroup$

1 Answer 1

1
$\begingroup$

You are correct: you get one hidden state per time-step. The hidden size, however, refers to the number of features that are used to compute each hidden state, and are therefore independent of your sequence length.

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .