If we use LSTMCell from torch:

  1. The initial hidden and cell layers should be CONSTANT (from the first time you run the program) and saved right? Like random seeds?

    class Model:
    def __init__(...):
        self.lstm_h0 = torch.randn(1, hidden_size, requires_grad=False)
        self.lstm_c0 = torch.randn(1, hidden_size, requires_grad=False)
  2. Let's say our problem is we want to count the number of balls in a video and run an LSTM over each frame of the video.

    self.lstm_cell = LSTMCell(...)
    for i in range(num_frames):
        h, c = self.lstm_cell(frame[i], (h, c))

    The final output embedding we use (to then project into a scalar 'ball count') is the final "h" hidden output, right?

  3. I have (N, nB, nE) dimensional data, call it "my_data". N is the batch dimension, nB is an object dimension, and nE is an embedding dimension. I want to use LSTM to perform dimensionality reduction over nB.

    for i in range(nB):
        h, c = self.lstm_cell(my_data[:, i, :], (h, c))

Is this the correct way to do it?

  • $\begingroup$ LSTMs can change dimensionality of $N_E$ not $N_B$ unless you plan to just take the final outtput vector which will be a pseudo aggregation of all the $N_E$ frames $\endgroup$ – mshlis Aug 23 '19 at 12:46
  • $\begingroup$ Yes, i want the aggregation of the nE frames. And I assume this can be much more expressive than a weighted average $\endgroup$ – user3180 Aug 23 '19 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.