Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

How Long Short Term Memory (LSTM) work for time series classification?

I first got the concept of LSTM on how it works word to word prediction etc. However, I want to know how it work with the time-series classification. For example I have the follwing data (see image ...
0
votes
0answers
14 views

How will actual labels be matched with predicted labels when LSTM discards data even from current time stamp input data?

I read the tutorial of LSTM from here. However, I have certain doubts that I need to address. Since we use true labels and do not remove anything from the original data, then how is it possible for ...
0
votes
1answer
30 views

Using a Neural Network (LSTM) to approve/reject word-type sequences

I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence. For instance: "Mike's Airplane" would output "...
0
votes
0answers
19 views

Time series forecasting for multiple objects with common features

I know the title of this question may raise an eyebrow, but I can't find the technical terms to define or investigate the actual problem. To demonstrate my problem with a simple hypothetical scenario: ...
2
votes
1answer
110 views

Price difference predictions curve almost vanished

With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well ...
1
vote
1answer
42 views

Why doesn't the LSTM model improve the time-series forecasting significantly with respect to the MLP model?

I have recently started learning time series forecasting. I have a dataset of the weekly payment history of 10k clients over 1 year, and I want to predict the future 5 payments for a test set of 1k ...
0
votes
0answers
13 views

End-to-end learning using LSTM-AE

I want to use prediction models like LSTM-AE to predict time-series data. The feature that the neural network should learn is in frequency between 40-60Hz. So, in order to learn the feature more ...
1
vote
1answer
63 views

time-series prediction : loss going down, then stagnates with very high variance

I am trying to design a model based on LSTM cells to do time-series prediction. The ouput value is an integer in [0,13]. I have noticed that one-hot encoding it and ...
1
vote
1answer
25 views

How can I address missing values for LSTM?

I'm a student and writing my first paper for submission on conference. I have a question there is a dataset below. this is temporal-spatial dataset. ...
0
votes
0answers
19 views

Keras MLP performing better than Transformers

I'm working on a comparative study using some models in a sentiment analysis task: MLPs and LSTMs with and without the use of word embeddings (GloVe and Word2Vec) and two Transformer-based models (...
6
votes
2answers
187 views

How to shorten the development time of a neural network?

I am developing an LSTM for sequence tagging. During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. After ...
2
votes
1answer
93 views

LSTM model on different time scales

I am a newbie to machine learning. I have an LSTM model that predicts the next output n+1 time 1, params 1, output 1 time 2, params 2, output 2 time 3, params 3, output 3 . . time n, params n, , ...
4
votes
2answers
524 views

How should we regularize an LSTM model?

There are five parameters from an LSTM layer for regularization if I am correct. To deal with overfitting, I would start with reducing the layers reducing the hidden units Applying dropout or ...
0
votes
0answers
5 views

Feature selection by Simple regression vs finite impulse response (FIR) method (on TIME Series analysis)

We are working on prediction one company production estimation and the main field of works is like stock market prediction(Time series analysis and process data). So I have some comment on using ...
2
votes
1answer
60 views

How do LSTM and GRU avoid to overcome the vanishing gradient problem?

I'm watching the video Recurrent Neural Networks (RNN) | RNN LSTM | Deep Learning Tutorial | Tensorflow Tutorial | Edureka where the author says that the LSTM and GRU architecture help to reduce the ...
0
votes
1answer
93 views

Why won't my model train with CTC loss?

I am trying to train an LSTM using CTC loss, but the loss does not decrease when I train it. I have created a minimal example of my issue by creating training data where the network simply has to copy ...
1
vote
0answers
24 views

LSTM Recursive Forecast

I am confused about the way the LSTM networks work when forecasting with a horizon that is not finite, but I'm rather searching for a prediction in whatever time in future. In physical terms, I would ...
0
votes
0answers
13 views

Seq2Seq Models not used for NLP - input to the Decoder?

I am looking into Seq2Seq models but using it to make multi-step predictions of factory data and I am getting a little confused with the inputs to the Decoder model. Correct me if I am wrong, but the ...
1
vote
0answers
10 views

LSTM Forecast Evolution

I have a confusion about the way the LSTM networks work when forecasting with an horizon that is not finite but I'm rather searching for a prediction in whatever time in future. In physical terms I ...
0
votes
1answer
19 views

Changing a CNN-LSTM image captioning architecture to use BiLSTMs

Currently I'm dealing with an assignment that made us implement the network mentioned in this paper. The network has an architecture similar to this: As you can see it uses a Unidirectional RNN (in ...
0
votes
0answers
36 views

What do RNN, LSTM, and GRU layers do in Tensorflow?

I have gone through some theoretical introductions of RNN and LSTM, which do not contain any code, and they describe in fair detail what the cells do, how they apply operations like forget, sigmoid, ...
0
votes
1answer
35 views

What is it about sigmoid activations in particular that allows for the keeping and forgetting of past information from different time scales?

My understanding is that normal recurrent neural networks (RNNs) are not good at keeping past information from different time scales. Furthermore, my understanding is that Gated RNNs, such as Long ...
0
votes
0answers
16 views

Neural Networks different architectures but similar training curves

I have a base neural network architecture for (3D) image sequences classification, made of conv layers followed by a LSTM and dense layers. I have 3 similar architectures : 3 Conv -> 1 LSTM -> ...
5
votes
2answers
579 views

What evaluation metric are used for sequence-to-sequence prediction problems?

I am solving many sequence-to-sequence prediction problems using RNN/LSTM. What type of evaluation metrics can be used for sequence prediction problems? One metric is the mean squared error (MSE) ...
0
votes
0answers
11 views

What does it mean when predicted results are constant values?

I'm practicing with some data with a LSTM neural nets to come up with predicted data, comparing with actual data. I generated an image to show what I came up with. The blue line is actual data, and ...
1
vote
0answers
31 views

Forecasting of spatio-temporal event data

I’m currently working on my dissertation which is centred around forecasting social conflict events. I’m using data from GDELT (Global Database of Events, Tone, and Language) to develop my forecasting ...
0
votes
1answer
185 views

Feeding YOLOv4 image data into LSTM layer?

How would one extract the feature vector from a given input image using YOLOv4 and pass that data into an LSTM to generate captions for the image? I am trying to make an image captioning software in ...
2
votes
1answer
222 views

How to train a LSTM model with multi dimensional data

I am trying to train my model using LTSM layer in Keras (python). I have some problems regarding the data representation and feeding it into the model. My data is 184 XY coodinates encoded as a numpy ...
1
vote
1answer
699 views

How to use the LSTM layer in PPO architecture?

What is the best way of using the LSTM layer in PPO architecture? Should I use them in the first layer of both actor and critic, or use them just before the final layer of these networks? Should I ...
13
votes
3answers
3k views

What is the relationship between the size of the hidden layer and the size of the cell state layer in an LSTM?

I was following some examples to get familiar with TensorFlow's LSTM API, but noticed that all LSTM initialization functions require only the num_units parameter, ...
3
votes
2answers
53 views

Problem extracting features from convolutional layer where the dimensions are big for feature maps

I have trained a convolutional neural network on images to detect emotions. Now I need to use the same network to extract features from the images and use them to train an LSTM. The problem is: the ...
1
vote
0answers
16 views

Is my dataset unlearnable, or is my LSTM model not smart enough?

I have time-series data obtained from a video. The data is composed of bitrate and corresponding label pairs for each timestamp: The distribution over the first 30 seconds is as follows: I have ...
1
vote
1answer
37 views

Rescaling time-series data with very spiky pattern for training data in LSTM network

I am working with some time-series hydrology data. Our goal is to forecast the time series forward, meaning predicting the data 1 month, 3 months ,6 months into the future. The data itself(image below)...
0
votes
0answers
43 views

Is it sensible to combine GRU/LSTM with the transformer's encoder?

Is it sensible to combine GRU/LSTM with the transformer's encoder? If we take the output of a GRU (uni or bi-directional), and then feed it as input to the transformer's encoder, would that help in ...
0
votes
0answers
14 views

Understanding metrics, understanding my LSTM results

I'm trying to learn about forecasting time-series methods, my first approach to achieve it is using LSTM. Lets suppose I have my data well processed, and I have my data time-series correctly. In that ...
1
vote
1answer
47 views

What kind of neural network should I build to classify each instance of a time series sequence?

Let's say I have the time-series dataset below-left. I would like to train a model in such a way that, if I feed the model with an input like the test sequence below, it should be able to classify ...
0
votes
1answer
59 views

How can Image Caption work?

I have two models and a file contains captions for images. The output of model 1 is .pkl files that contain the features of the images. Model 2 is the language model that will be trained with the ...
0
votes
0answers
36 views

What is the role of this initial state of the LSTM layer in an encoder of a seq2seq model?

I am trying to follow this guide to implement a seq2seq machine translation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention. The tutorial's ...
1
vote
1answer
362 views
1
vote
1answer
38 views

Recommended Time serie forecasting model for Fibonacci levels classification

I have a set of time series data which gives me fibonacci levels and the duration at which the value is at this level. Data structure to look like: ...
0
votes
0answers
16 views

Which is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?

Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch. ...
0
votes
1answer
91 views

Seq2Seq model produces repeating words

My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
2
votes
2answers
143 views

Extract features with CNN and pass as sequence to RNN

I read an article about captioning videos and I want to use solution number 4 (extract features with a CNN, pass the sequence to a separate RNN) in my own project. But for me, it seems really strange ...
3
votes
2answers
169 views

Price Movement Forecasting Issue

I am working on a project for price movement forecasting and I am stuck with poor quality predictions. At every time-step I am using an LSTM to predict the next 10 time-steps. The input is the ...
4
votes
1answer
653 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training an LSTM network with multiple inputs and several LSTM layers in order to set up a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation ...
0
votes
0answers
14 views

Why is my LSTM model predicting accurately for only a few values and showing drastic aberration later?

I am training an LSTM model using stock data for time series forecasting and the results are a little confusing to me. This is the prediction I get after 5 epochs. And this after 100 epochs. Why the ...
1
vote
1answer
242 views

Are 1D CNNs really the appropriate model for human activity recognition?

This article on human activity recognition states that 1D convolutional neural networks work the best on the classification of human activities using data from the accelerometer. But I think that ...
0
votes
0answers
20 views

How to Classify Game Stages Based on Bitrate Time Series Data Using RNN - LSTM

I need suggestions for my project and would be glad if you would give me a hand. I have a dataset of frames obtained from the old-school game DOOM. Each frame in the dataset has the following columns: ...

1
2 3 4 5 6