Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?

Question

I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it has a many to one shape.

For example, let's say we're at timestep n, and the following timeseries is part of my input:

[[100, 10], [300, 30], [200, 20]]

And it maps to some output, let's say 1

Great. But let's say at timestep n - 1, when the input was just [[100, 10], [300, 30]], the output was 0. How will the LSTM know this?

Should I include the same data at different timesteps (using something like zero padding) with the corresponding output? Or am I totally misunderstanding something about how LSTMs work?

Answer 1

The hidden and carry states of an LSTM contain the current 'embedding' of the past data that has been passed through the cell. The hidden state is also taken as output at each timestep. If, as you are writing, make the hidden state of size 1, then indeed it will not remember much of the past.

However, what is more common, is to have a larger number of hidden neurons in the LSTM and then connect a fully connected dense layer to the output, which takes in this hidden state (embedding) and transforms it to a single output node. That way you can remember a lot of data, but still have a 1 neuron output.