# Initial LSTM hidden state and cell

If we use LSTMCell from torch:

1. The initial hidden and cell layers should be CONSTANT (from the first time you run the program) and saved right? Like random seeds?

class Model:

def __init__(...):
self.lstm_h0 = torch.randn(1, hidden_size, requires_grad=False)
self.lstm_c0 = torch.randn(1, hidden_size, requires_grad=False)

2. Let's say our problem is we want to count the number of balls in a video and run an LSTM over each frame of the video.

self.lstm_cell = LSTMCell(...)

for i in range(num_frames):
h, c = self.lstm_cell(frame[i], (h, c))


The final output embedding we use (to then project into a scalar 'ball count') is the final "h" hidden output, right?

3. I have (N, nB, nE) dimensional data, call it "my_data". N is the batch dimension, nB is an object dimension, and nE is an embedding dimension. I want to use LSTM to perform dimensionality reduction over nB.

for i in range(nB):
h, c = self.lstm_cell(my_data[:, i, :], (h, c))


Is this the correct way to do it?

• LSTMs can change dimensionality of $N_E$ not $N_B$ unless you plan to just take the final outtput vector which will be a pseudo aggregation of all the $N_E$ frames – mshlis Aug 23 at 12:46
• Yes, i want the aggregation of the nE frames. And I assume this can be much more expressive than a weighted average – user3180 Aug 23 at 13:24