Is a recurrent layer same as LSTM or single-layered LSTM?

Question

In MLP, there are neurons that form a layer. Each hidden layer gives a vector of number that is the output of that layer.

In CNN, there are kernels that form a convolutional layer. Each layer gives feature maps that are the output of that layer.

In LSTM, there are cells that form a recurrent layer. Each layer gives a sequence that is the output of that layer.

This is my understanding of the basic terminology regarding MLP, CNN, and LSTM.

But consider the following description regarding the number of layers in LSTM in PyTorch

num_layers – Number of recurrent layers. E.g., setting num_layers=2 would mean stacking two LSTMs together to form a stacked LSTM, with the second LSTM taking in outputs of the first LSTM and computing the final results. Default: 1

The description uses the "number of recurrent layers" and the "LSTM" in a similar manner. How I can understand this? Is it costmary to consider a recurrent layer as an LSTM?

Answer 1

Depending on the context, when people use the term LSTM, they either refer to

• an LSTM layer,
• an LSTM unit (like a recurrent unit in an RNN or neuron in an MLP), or
• an LSTM neural network (i.e. an RNN that uses LSTM units or layers).

In TensorFlow, an LSTM is a layer, so you can stack multiple LSTMs to create deeper architectures.

In PyTorch, the class LSTM can create an LSTM layer or multiple LSTM layers stacked together. You also have an LSTMCell, which should be just one LSTM layer.

My answer here should also be useful.