Number of LSTM layers needed to learn a certain number of sequences

Question

Theoretically, number of units for a LSTM layer is the number of hidden states or the max length of sequences as per my practice.

For example, in Keras:

Lstm1 = LSTM(units=MAX_SEQ_LEN, return_sequences=False);

However, with lots of sequences to train, should I add more LSTM layers? because increasing MAX_SEQ_LEN is not the way as it doesn't help make the network better since the extra number of hidden states isn't useful any more.

I'm considering increasing number of LSTM layers, but how many are enough?

For example, 3 of them:

Lstm1 = LSTM(units=MAX_SEQ_LEN, return_sequences=True);
Lstm2 = LSTM(units=MAX_SEQ_LEN, return_sequences=True);
Lstm3 = LSTM(units=MAX_SEQ_LEN, return_sequences=False);

Answer 1

From my personal experience, the units hyperparam in LSTM is not necessary to be the same as max sequence length. Add more units to have the loss curve dive faster.

And about the number of LSTM layers, trying out a single LSTM layer is a good start point, the model trains better with more LSTM layers.

For example, MAX_SEQ_LEN=10, in Keras:

Lstm1 = LSTM(units=30, return_sequences=True); #Time sequence, to feed to next layer
Lstm2 = LSTM(units=20, return_sequences=True); #Time sequence, to feed to next layer
Lstm3 = LSTM(units=10, return_sequences=False);

Output = Lstm3(Lstm2(Lstm1));
```