Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

Filter by
Sorted by
Tagged with
0
votes
0answers
28 views

Is it sensible to combine GRU/LSTM with the transformer's encoder?

Is it sensible to combine GRU/LSTM with the transformer's encoder? If we take the output of a GRU (uni or bi-directional), and then feed it as input to the transformer's encoder, would that help in ...
0
votes
0answers
11 views

Understanding metrics, understanding my LSTM results

I'm trying to learn about forecasting time-series methods, my first approach to achieve it is using LSTM. Lets suppose I have my data well processed, and I have my data time-series correctly. In that ...
0
votes
0answers
33 views

What is the role of this initial state of the LSTM layer in an encoder of a seq2seq model?

I am trying to follow this guide to implement a seq2seq machine translation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention. The tutorial's ...
1
vote
1answer
47 views

What kind of neural network should I build to classify each instance of a time series sequence?

Let's say I have the time-series dataset below-left. I would like to train a model in such a way that, if I feed the model with an input like the test sequence below, it should be able to classify ...
1
vote
1answer
38 views

Recommended Time serie forecasting model for Fibonacci levels classification

I have a set of time series data which gives me fibonacci levels and the duration at which the value is at this level. Data structure to look like: ...
0
votes
0answers
16 views

Which is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?

Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch. ...
0
votes
1answer
44 views

Seq2Seq model produces repeating words

My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
0
votes
0answers
10 views

Why is my LSTM model predicting accurately for only a few values and showing drastic aberration later?

I am training an LSTM model using stock data for time series forecasting and the results are a little confusing to me. This is the prediction I get after 5 epochs. And this after 100 epochs. Why the ...
0
votes
0answers
20 views

How to Classify Game Stages Based on Bitrate Time Series Data Using RNN - LSTM

I need suggestions for my project and would be glad if you would give me a hand. I have a dataset of frames obtained from the old-school game DOOM. Each frame in the dataset has the following columns: ...
0
votes
1answer
43 views

Advantages of CNN vs. LSTM for sequence data like text or log-files

When do you tend to use CNN rather than LSTM (or the other way round) in classification or generation tasks of sequential data like text or log-data? What are the reasons for the decision and what ...
1
vote
1answer
25 views

How to train an LSTM to classify based on rare historic event?

I want an LSTM to output one of two classes (Y, N), per frame, based on all the input so far. My original inputs are very long (~100000 samples long, far more than a standard LSTM training can handle ...
0
votes
0answers
17 views

Converting inputs as a batch for time series classification would increase accuracy?

I have sensor dataset. I have already classified these data with LSTMs.I have a dataframe with 2 features and a class column. Assume that I take every two rows(inputs) respectively and make the ...
0
votes
0answers
23 views

PyTorch: LSTM error while trying to update the hidden state

I am trying to train an LSTM while keeping its hidden state (LSTM stateful) until the moment when I am going to start a new epoch(episode). But here it's come an interesting situation because I am ...
0
votes
0answers
29 views

PyTorch: How to deal with hidden states of an LSTM?

I have a time series in which each date is correlated with the preview one, and base on that I am trying to predict action 1 and action 2. But the problem is that I am not sure how to deal with the <...
0
votes
1answer
46 views

Must all CNNs and RNNs not have a fully connected layer in order to be considered as such?

In the paper Wrist-worn blood pressure tracking in healthy free-living individuals using neural networks, the authors talk about a combination of feed-forward and recurrent layers, as if FC layers ...
1
vote
1answer
31 views

Do RNNs/LSTMs really need to be sequential?

There are many articles comparing RNNs/LSTMs and the Attention mechanism. One of the disadvantages of RNNs that is often mentioned is that while Attention can be computed in parallel, RNNs are highly ...
1
vote
1answer
42 views

Can RNNs get inputs and produce outputs similar to the inputs and outputs of FFNNs?

RNN and LSTM models have many interesting architectures that can be modified in various ways. We can also compose their input and output data in quite interesting ways. However, in the examples that I ...
0
votes
1answer
56 views

How can Image Caption work?

I have two models and a file contains captions for images. The output of model 1 is .pkl files that contain the features of the images. Model 2 is the language model that will be trained with the ...
0
votes
0answers
17 views

Multi dimensional LSTM modeling in KERAS

I have a database of time series signals with multiple features and Im trying to build a model to predict whether or not two samples are related to each other. For example : a database of 1000 sample ...
0
votes
0answers
8 views

Is it possible to get this loss with spikes, when training an LSTM with the cross-entropy on a multi-class classification problem of a time series?

The main question here will be "should I look for a bug?" My setup is a time series multiclass classification task, labeled per frame. I am using an LSTM, feeding inputs and using ...
0
votes
1answer
31 views

Convert LSTM univariate Autoencoder to multivariate Autoencoder

I have the following code snippet which takes in a single column of value i.e. 1 feature. How do I modify the LSTM model such that it accepts 3 features? ...
1
vote
1answer
22 views

What is the input to the left most LSTM cell c(t-1) and h(t-1)?

Given an LSTM model with 3 cells shown below, what would be the input to the left most cell c(t-1) and h(t-1)?
2
votes
1answer
52 views

Is this LSTM layer learning anything?

I've trained a CNN-LSTM model but the results weren't satisfactory, so I took a look at my weight distributions and this is what I got: I don't understand. Is this layer learning anything? Or no? ...
1
vote
1answer
31 views

How to define a “don't care” class in time series classification in Pytorch?

This is a theoretical question. Setup I have a time series classification task in which I should output a classification of 3 classes for every time stamp t. All ...
1
vote
0answers
20 views

Given the word embeddings, how do I create the sentence composed of the corresponding words?

I have done some reading. I want to implement an LSTM with pre-trained word embeddings (I also have plans to create my word embeddings, but let's cross that bridge when we come to it). In any given ...
2
votes
0answers
23 views

If I want to predict two unrelated values given the same sequence of data points, should I have a model with two outputs or two models?

I want to predict two separate y-values (not really logically connected) based on an input sequence of data (values x). Using LSTM cells. Should I train two models separately or should I just increase ...
0
votes
0answers
23 views

How to make an LSTM ensemble model with different input shapes

This is what I got so far for making an lstm ensemble with one model input for each of the lstm models and for the ensemble model and it works perfectly. ...
1
vote
0answers
42 views

How to improve prediction performance of periodic data?

I have a 1 column dataset of $50 000$ points where 95% of the values equal $-50$. The data looks like the following: $$\begin{matrix} \text{time} & \text{value}\\ 1&-50 \\ 2&-50 \\ 3&-...
0
votes
1answer
80 views

Is it possible to predict $x^2$, $\log(x)$, or variable function of $x$ using RNN?

There were some posts that using RNN can predict the next point of the sine wave function with data history. However, I wondered if it also works on all the functions of $x$, such as $x^2$, $x^3$, $\...
0
votes
0answers
37 views

Seq2Seq Modelling: when implementing some machine translation net, how are special tokens embedded?

When implementing any encoder-decoder network for machine translation, during training we provide the true output sentence to the decoder so that the context vector (from source language) may be ...
0
votes
0answers
79 views

How to make an ensemble model of two LSTM models with different window sizes i.e. different data shapes

Below is the Python code for making an ensemble model. All the inputs are the same for all three models. But what if the models have different input shapes due to different window size, such as LSTM ...
1
vote
0answers
23 views

Using LSTM model to train spatial inputs

I have an $x$-$y$ plane, inside that plane I have 9 paths $(p_1, p_2, \dots, p_3)$. Each path is classified into one of the three classes $(c_1, c_2, c_3)$. Each path has 100 coordinates points i.e $((...
4
votes
2answers
175 views

How should we regularize an LSTM model?

There are five parameters from an LSTM layer for regularization if I am correct. To deal with overfitting, I would start with reducing the layers reducing the hidden units Applying dropout or ...
0
votes
1answer
47 views

Is the working of RNNs, LSTM and GRU sequential or parallel?

You take any blog or any example and all they tell you about is the given picture below. It has 4 different matrices and 3 of whose weights are shared. So, I'm wondering how is this achieved in ...
0
votes
0answers
13 views

Is it possible and if so does it make sense to have dense layers in between LSTM layers?

I am new to LSTMs and I was wondering if it is possible to have LSTM layer then dense then LSTM again and does it make sense?
0
votes
0answers
15 views

Text generation with LSTM with multiple correlated inputs

I am currently working on a music-generation project, inspired by an already existing project called Deepbach. My dataset are the Bach chorales, which are all composed of 4 independent (but related) ...
0
votes
1answer
53 views

Why won't my model train with CTC loss?

I am trying to train an LSTM using CTC loss, but the loss does not decrease when I train it. I have created a minimal example of my issue by creating training data where the network simply has to copy ...
0
votes
0answers
23 views

During Backpropagation in LSTM, why is the previous output $h_{t-1}$ considered constant w.r.t any $W$ while computing derivative?

I've just started learning LSTM, and some points in the process of calculating the gradients are getting me confused. Say, for example, we want to compute $\frac{\partial}{\partial W_i}L$, where $L$ ...
0
votes
0answers
99 views

How are temporal links made between following sequences in RNN?

Say I use an RNN, whatever is the cell's type, to perform time series classification. It can thus be seen as sequence classification. The time series is split into random, equal size, overlapping ...
0
votes
0answers
9 views

Is the information in the hiddenstate of a RNN worth processing further after the input passes the RNN?

I hope the question is understandable. I just wanted to ask if the hidden state, which is passed through the timesteps/cells of an RNN/LSTM/GRU to deliver information from $\text{cell}_{i-1}$ to the $\...
0
votes
0answers
78 views

Understanding Stable Baselines Custom Policies

I'm trying to understand the structure of the custom recurrent policy introduced in the documentation of the Stable Baselines: How exactly is the Lstm NN constructed? (check code below) From what I ...
0
votes
0answers
19 views

Using an LSTM for model-based RL in a POMDP

I am trying to set up an experiment where an agent is exploring an n x n gridworld environment, of which the agent can see some fraction at any given time step. I'd like the agent to build up some ...
2
votes
1answer
36 views

Understanding LSTM through example

I want to code up one time step in a LSTM. My focus is on understanding the functioning of the forget gate layer, input gate layer, candidate values, present and future cell states. Lets assume that ...
0
votes
0answers
60 views

Time series prediction using LSTM and CNN-LSTM: which is better?

I am working on LSTM and CNN to solve the time series prediction problem. I have seen some tutorial examples of time series prediction using CNN-LSTM. But I don't know if it is better than what I ...
2
votes
0answers
37 views

Can One-Hot Vectors be used as Inputs for Recurrent Neural Networks?

When using an RNN to encode a sentence, one normally takes each word, passes it through an embedding layer, and then uses the dense embedding as the input into the RNN. Lets say instead of using dense ...
2
votes
1answer
28 views

Does this diagram represent several LSTMs, or one through several timesteps?

I'm trying to read this paper describing Google's LSTM architecture for machine translation. It features this diagram on page 4: I'm interested in the encoder block, on the left. Apparently, the pink ...
1
vote
0answers
29 views

How does Google's 2016 GNMT architecture work?

I'm trying to read this paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me. This is a picture ...
1
vote
0answers
45 views

How is input defined for a biaxial lstm network for generating music?

I am reading Composing Music With Recurrent Neural Networks by Daniel D. Johnson. But I am really confused about the input passed to this network. If we pass notes of music along the time axis, then ...
0
votes
0answers
18 views

How to afine the extremity values in regression prediction with Keras?

I made a stack of bidirectional LSTM layers following by Dense layers (with swish activation functions) in order to predict a continuous value between 0 and 2. I compiled the model with ...
3
votes
1answer
60 views

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously?

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously? Is there an agreed up protocol on how to extract this information? What ...

1
2 3 4 5 6