Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

How sensitive are LSTM's to random zero values in its target feature when training?

I have worked with lstm's in the past, specifically for time series forecasting. However, the target feature in these time series were relatively "stable". With the loosely defined "...
0
votes
0answers
7 views

Categorical Location based Time Series data Prediction using LSTMs

I have some time series data with ActivityType as location shared below. Each CaseID has 6 unique values with different ...
0
votes
0answers
25 views

Is this a good implementation of this LSTM architecture?

I had been looking at some OCR problems and came across this presentation. I implemented it. In the presentation, there is the LSTM-Stack (diagram and algorithm, slide 32): Here is a visualization of ...
0
votes
0answers
18 views

How Long Can BPTT Truncated?

I wanted to ask what is, in general, the maximum value (the order of magnitude) of the number of time steps I can back-propagate in the past using TBTT (Truncated Backpropagation Through Time) in an ...
0
votes
0answers
16 views

Boundedness of cell states in MC-LSTM (mass-conserving LSTM)

I'm currently reading Hoedt et al's paper on mass-conserving LSTM. In the corollary it is stated that "[T]he memory cells, $c_k^\tau$, are bounded by the sum of mass inputs $\sum_{t=1}^\tau x^t+...
1
vote
0answers
23 views

Machine learning with raw data alone / or raw data with its statistics

My question is very general and it does not originate from a specific problem. Let's assume that, through experience, we have learned that some statistical property of a set of data is important in ...
7
votes
1answer
757 views

Is LSTM a subcategory of RNN?

Is the LSTM-Architecture a subcategory of RNNs? Or are they totally different? Literature doesn't seem to be unitary on this. This figure appears to explain the models to be alternatives, but I ...
0
votes
1answer
23 views

Do LSTM in tensorflow work sequentially or in parallel

I have a basic understanding how a cell and a layer of an LSTM works. However, I get confused by what "number of units" (as termed in tensorflow) exactly means. A unit is, as far as I ...
0
votes
1answer
41 views

Reinforcing Learning when action has no effect on the environment

I am trying to get my head around a problem where the action by the agent can not change the environment. Without going into details, my problem is about error correction in an stochastic environment. ...
2
votes
1answer
27 views

What type of neural network do you need if you want to detect an action or dynamic pattern instead of a static pattern?

Let's say that you want to detect if a man is running, walking, or dancing instead of just detecting a man still. What type of neural networks will you use for this purpose?
0
votes
0answers
34 views

What's the best way to feed stories to a neural network?

I'm trying to train a model that would generate stories. I have a dataset of 2000 stories prepared. They are tokenized and one-hot encoded. I can't load them all at once as a one big dataset, because ...
1
vote
0answers
14 views

The results are not correct when predicting the future for a very long period of time with LSTM

I am currently using LSTM to try to predict future data in AirPassengers.csv. This is current code op my Colab (sorry for the comments are Japanese) https://colab.research.google.com/drive/...
0
votes
0answers
14 views

Time series forecasting with some challenges

I'm attempting to devise a strategy to make time series forecasts based on costs accumulated over time. My dataset contains about 7500 time-series sequences (call it an instance for now), each having ...
0
votes
2answers
24 views

What is the reason for a training loss that drops but validation that NEVER does

I've been working on learning about NLP via a beginners competition on Kaggle. I first trained a model with an embedding layer and then a simple linear layer. I actually got way better than a flip of ...
0
votes
0answers
31 views

LSTM and CNN - feature engineering and order for time series classification

My questions are related to multivariate time series classification, hence it may differ from forecasting problems. I can have either variable (entire history of the series) or fixed time steps (...
0
votes
0answers
10 views

LSTM predictions are one time step lagging

My problem involves electricity prediction (time-series problem) for 1-hour ahead. I am using LSTM to forecast. Length of Dataset: 1 year at one-hour interval Input: Outdoor Temperature (Ot), ...
0
votes
0answers
29 views

How Long Short Term Memory (LSTM) work for time series classification?

I first got the concept of LSTM on how it works word to word prediction etc. However, I want to know how it work with the time-series classification. For example I have the follwing data (see image ...
0
votes
0answers
18 views

How will actual labels be matched with predicted labels when LSTM discards data even from current time stamp input data?

I read the tutorial of LSTM from here. However, I have certain doubts that I need to address. Since we use true labels and do not remove anything from the original data, then how is it possible for ...
0
votes
0answers
25 views

Time series forecasting for multiple objects with common features

I know the title of this question may raise an eyebrow, but I can't find the technical terms to define or investigate the actual problem. To demonstrate my problem with a simple hypothetical scenario: ...
0
votes
1answer
52 views

Using a Neural Network (LSTM) to approve/reject word-type sequences

I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence. For instance: "Mike's Airplane" would output "...
0
votes
1answer
16 views
0
votes
0answers
13 views

End-to-end learning using LSTM-AE

I want to use prediction models like LSTM-AE to predict time-series data. The feature that the neural network should learn is in frequency between 40-60Hz. So, in order to learn the feature more ...
1
vote
1answer
28 views

How can I address missing values for LSTM?

I'm a student and writing my first paper for submission on conference. I have a question there is a dataset below. this is temporal-spatial dataset. ...
1
vote
1answer
46 views

Why doesn't the LSTM model improve the time-series forecasting significantly with respect to the MLP model?

I have recently started learning time series forecasting. I have a dataset of the weekly payment history of 10k clients over 1 year, and I want to predict the future 5 payments for a test set of 1k ...
0
votes
0answers
22 views

Keras MLP performing better than Transformers

I'm working on a comparative study using some models in a sentiment analysis task: MLPs and LSTMs with and without the use of word embeddings (GloVe and Word2Vec) and two Transformer-based models (...
0
votes
0answers
7 views

Feature selection by Simple regression vs finite impulse response (FIR) method (on TIME Series analysis)

We are working on prediction one company production estimation and the main field of works is like stock market prediction(Time series analysis and process data). So I have some comment on using ...
2
votes
0answers
31 views

LSTM Recursive Forecast

I am confused about the way the LSTM networks work when forecasting with a horizon that is not finite, but I'm rather searching for a prediction in whatever time in future. In physical terms, I would ...
1
vote
0answers
16 views

Seq2Seq Models not used for NLP - input to the Decoder?

I am looking into Seq2Seq models but using it to make multi-step predictions of factory data and I am getting a little confused with the inputs to the Decoder model. Correct me if I am wrong, but the ...
1
vote
0answers
10 views

LSTM Forecast Evolution

I have a confusion about the way the LSTM networks work when forecasting with an horizon that is not finite but I'm rather searching for a prediction in whatever time in future. In physical terms I ...
0
votes
0answers
42 views

What do RNN, LSTM, and GRU layers do in Tensorflow?

I have gone through some theoretical introductions of RNN and LSTM, which do not contain any code, and they describe in fair detail what the cells do, how they apply operations like forget, sigmoid, ...
0
votes
1answer
22 views

Changing a CNN-LSTM image captioning architecture to use BiLSTMs

Currently I'm dealing with an assignment that made us implement the network mentioned in this paper. The network has an architecture similar to this: As you can see it uses a Unidirectional RNN (in ...
0
votes
1answer
37 views

What is it about sigmoid activations in particular that allows for the keeping and forgetting of past information from different time scales?

My understanding is that normal recurrent neural networks (RNNs) are not good at keeping past information from different time scales. Furthermore, my understanding is that Gated RNNs, such as Long ...
0
votes
0answers
17 views

Neural Networks different architectures but similar training curves

I have a base neural network architecture for (3D) image sequences classification, made of conv layers followed by a LSTM and dense layers. I have 3 similar architectures : 3 Conv -> 1 LSTM -> ...
0
votes
0answers
11 views

What does it mean when predicted results are constant values?

I'm practicing with some data with a LSTM neural nets to come up with predicted data, comparing with actual data. I generated an image to show what I came up with. The blue line is actual data, and ...
1
vote
0answers
35 views

Forecasting of spatio-temporal event data

I’m currently working on my dissertation which is centred around forecasting social conflict events. I’m using data from GDELT (Global Database of Events, Tone, and Language) to develop my forecasting ...
1
vote
0answers
19 views

Is my dataset unlearnable, or is my LSTM model not smart enough?

I have time-series data obtained from a video. The data is composed of bitrate and corresponding label pairs for each timestamp: The distribution over the first 30 seconds is as follows: I have ...
1
vote
1answer
55 views

Rescaling time-series data with very spiky pattern for training data in LSTM network

I am working with some time-series hydrology data. Our goal is to forecast the time series forward, meaning predicting the data 1 month, 3 months ,6 months into the future. The data itself(image below)...
0
votes
0answers
52 views

Is it sensible to combine GRU/LSTM with the transformer's encoder?

Is it sensible to combine GRU/LSTM with the transformer's encoder? If we take the output of a GRU (uni or bi-directional), and then feed it as input to the transformer's encoder, would that help in ...
0
votes
0answers
16 views

Understanding metrics, understanding my LSTM results

I'm trying to learn about forecasting time-series methods, my first approach to achieve it is using LSTM. Lets suppose I have my data well processed, and I have my data time-series correctly. In that ...
0
votes
0answers
43 views

What is the role of this initial state of the LSTM layer in an encoder of a seq2seq model?

I am trying to follow this guide to implement a seq2seq machine translation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention. The tutorial's ...
1
vote
1answer
48 views

What kind of neural network should I build to classify each instance of a time series sequence?

Let's say I have the time-series dataset below-left. I would like to train a model in such a way that, if I feed the model with an input like the test sequence below, it should be able to classify ...
1
vote
1answer
40 views

Recommended Time serie forecasting model for Fibonacci levels classification

I have a set of time series data which gives me fibonacci levels and the duration at which the value is at this level. Data structure to look like: ...
0
votes
0answers
17 views

Which is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?

Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch. ...
0
votes
1answer
130 views

Seq2Seq model produces repeating words

My framework is an encoder-decoder (LSTM-to-LSTM) model, similar to this post. The model basically reads a sentence and generate another sentence. But, the thing is, after a few epochs training, the ...
0
votes
0answers
14 views

Why is my LSTM model predicting accurately for only a few values and showing drastic aberration later?

I am training an LSTM model using stock data for time series forecasting and the results are a little confusing to me. This is the prediction I get after 5 epochs. And this after 100 epochs. Why the ...
0
votes
0answers
20 views

How to Classify Game Stages Based on Bitrate Time Series Data Using RNN - LSTM

I need suggestions for my project and would be glad if you would give me a hand. I have a dataset of frames obtained from the old-school game DOOM. Each frame in the dataset has the following columns: ...
0
votes
1answer
45 views

Advantages of CNN vs. LSTM for sequence data like text or log-files

When do you tend to use CNN rather than LSTM (or the other way round) in classification or generation tasks of sequential data like text or log-data? What are the reasons for the decision and what ...
1
vote
1answer
26 views

How to train an LSTM to classify based on rare historic event?

I want an LSTM to output one of two classes (Y, N), per frame, based on all the input so far. My original inputs are very long (~100000 samples long, far more than a standard LSTM training can handle ...
0
votes
0answers
17 views

Converting inputs as a batch for time series classification would increase accuracy?

I have sensor dataset. I have already classified these data with LSTMs.I have a dataframe with 2 features and a class column. Assume that I take every two rows(inputs) respectively and make the ...
0
votes
0answers
75 views

PyTorch: LSTM error while trying to update the hidden state

I am trying to train an LSTM while keeping its hidden state (LSTM stateful) until the moment when I am going to start a new epoch(episode). But here it's come an interesting situation because I am ...

1
2 3 4 5 6