Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

Filter by
Sorted by
Tagged with
1 vote
0 answers
12 views

How to identify important features in data?

I have a couple opportunities to write a paper, or papers over some of the neural networks I have made. I was wondering if there are anyways to figure out why the neural network classifies the data I ...
user avatar
1 vote
1 answer
23 views

Does LSTM provide any unique value or advantages compared to other algorithms, including "vanilla" RNN?

I have heard a lot of hype around LSTM for all kinds of time-series based applications including NLP. Despite this, I haven't seen many (if any) applications of LSTM where LSTM performs uniquely well ...
user avatar
0 votes
0 answers
11 views

Finding "look_back" & "look_ahead" hyper-parameters for Seq2Seq models

For Seq2Seq deep learning architectures, viz., LSTM/GRU and multivariate, multistep time series forecasting, it is important to convert the data to a 3D dimension: (batch_size, look_back, ...
user avatar
  • 185
1 vote
1 answer
33 views

What is the advantage of adding CNN to LSTM for forecasting sequential data?

I am working with simulated sequential data and the goal is to forecast that data. Long-short-term-memory (LSTM) is one of the most advanced models to forecast time series according to this post. I ...
user avatar
  • 111
-1 votes
1 answer
36 views

How Many Hidden Units in an LSTM? [duplicate]

Is there any rule of thumb for choosing the number of hidden units in an LSTM? Is it similar to hidden neurons in a regular feedforward neural network? I'm getting better results with my LSTM when I ...
user avatar
  • 11
0 votes
1 answer
104 views

What is the difference between CNN-LSTM and RNN?

I'm starting to study RNN for a project of video prediction, but I encounter these CNN-LSTM models. Initially, I thought that is another name for RNN, but I think I get it wrong. Since I'm a beginner ...
user avatar
0 votes
1 answer
48 views

Do the values over 0.5 mean my model classified the data as a "1" label and vice versa?

I am doing binary classification using an LSTM and my output layer is 1 neuron with a sigmoid function. My labels are either 0 or 1. ...
user avatar
0 votes
1 answer
109 views

Training strategy on continuous video stream with CNN-LSTM

I have videos that are each about 30-40 mins long. With the first 5-10 mins (at 60fps, can be down-sampled to 5fps) are one type of activity that would be categorized by label-1 and the rest of the ...
user avatar
  • 101
0 votes
0 answers
45 views

LSTM: Simple value series vs Complex value series

A model for learning a trend graph can be this way: To learn a sequence of N numbers LSTM layer of M units Dense output node of 1 unit The problem is a trend graph in history can be simple: Case 1:...
user avatar
  • 1,183
0 votes
0 answers
24 views

Can you train LSTM on a dataset with several separate time-series?

I want to use LSTM in the problem of sports prediction. I know you can use LSTM to predict time-series values such as financial data ... In such time-series each value is part of the same sequence. In ...
user avatar
0 votes
0 answers
53 views

Neural network always learns to reproduce the input

I am trying to train a Convolutional LSTM network from scratch and it is always learning to reproduce the given input. I am trying to understand why this is happening and change this behavior. Could ...
user avatar
  • 9
1 vote
1 answer
182 views

Sequence Embedding using embedding layer: how does the network architecture influence it?

I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there ...
user avatar
0 votes
1 answer
26 views

time series analysis: predict number and type of service

I have temporal data regarding the number of customers who requested a specific service in a given period (month and year). Below is a small excerpt from the dataset: Month-year: month and year when ...
user avatar
  • 65
0 votes
0 answers
43 views

LSTM performance strictly decreases with sequence length input

I'm working on an event binary classification problem. More specifically, for a given event E I know some info about the event itself just before it's supposed to happen, encoded in an embedding ...
user avatar
0 votes
0 answers
29 views

LSTM - What kind of data should contain every dimension of input LSTM matrix, where does specific dimension points to?

I am a beginner and I have a hard time understanding inputs and outputs of LSTM. So from the begining, I am trying to create multivariate input&output LSTM for time series forecasting. Thankfully ...
user avatar
0 votes
1 answer
65 views

Is my intuition about RNN wrong?

Until today, my intuition about RNN (LSTM/GRU) was that this is some kind of NN that can remember previous inputs. Consider a task where you need to predict 0 if the previous input was 1. For example: ...
user avatar
  • 109
1 vote
1 answer
422 views

What is the time complexity for testing a stacked LSTM model?

In the data preparation phase, we have to divide the dataset into two parts: the training dataset and the test dataset. I have seen this post regarding the time complexity for training a model. ...
user avatar
-1 votes
1 answer
55 views

Is my dataset a time series dataset? and should I use an LSTM?

I have a dataset where I am recording temperature after every 4milliseconds till 500 and another feature "conductivity value". The length of the dataset is around a 1000 rows. I need to find ...
user avatar
1 vote
1 answer
101 views

Is a recurrent layer same as LSTM or single-layered LSTM?

In MLP, there are neurons that form a layer. Each hidden layer gives a vector of number that is the output of that layer. In CNN, there are kernels that form a convolutional layer. Each layer gives ...
user avatar
  • 3,099
1 vote
1 answer
34 views

What kind of NN to use to find misprints in test

I have a bunch of unique full names of users. I made pseudo-physical model to emulate misprints of desktop and mobile users (hence, fatfingering, jumpy fingers, accidentals touches of touch bar etc.) ...
user avatar
0 votes
0 answers
93 views

Which ML algorithm is the best for predict the next PRNG generated numbers?

I have a homework. The task is to decide, if the PRNG generated lottery is attackable/crackable or not. Details: Lottery: There is a lottery game where you have to choose 8 numbers between 1-20 for ...
user avatar
  • 1
2 votes
1 answer
460 views

Why is there tanh(x)*sigmoid(x) in a LSTM cell?

CONTEXT I was wondering why there are sigmoid and tanh activation functions in an LSTM cell. My intuition was based on the flow of tanh(x)*sigmoid(x) and the ...
user avatar
2 votes
0 answers
106 views

Why does the number of input tokens to an LSTM have an impact on the convergence of Integrated Gradients?

Background I am computing the attribution scores for a simple LSTM model using Integrated Gradients. This method defines the contribution of a feature to a model prediction by integrating over the ...
user avatar
  • 71
0 votes
0 answers
15 views

How sensitive are LSTM's to random zero values in its target feature when training?

I have worked with lstm's in the past, specifically for time series forecasting. However, the target feature in these time series were relatively "stable". With the loosely defined "...
user avatar
0 votes
0 answers
23 views

Categorical Location based Time Series data Prediction using LSTMs

I have some time series data with ActivityType as location shared below. Each CaseID has 6 unique values with different ...
user avatar
0 votes
0 answers
39 views

Is this a good implementation of this LSTM architecture?

I had been looking at some OCR problems and came across this presentation. I implemented it. In the presentation, there is the LSTM-Stack (diagram and algorithm, slide 32): Here is a visualization of ...
user avatar
0 votes
0 answers
19 views

How Long Can BPTT Truncated?

I wanted to ask what is, in general, the maximum value (the order of magnitude) of the number of time steps I can back-propagate in the past using TBTT (Truncated Backpropagation Through Time) in an ...
user avatar
0 votes
0 answers
18 views

Boundedness of cell states in MC-LSTM (mass-conserving LSTM)

I'm currently reading Hoedt et al's paper on mass-conserving LSTM. In the corollary it is stated that "[T]he memory cells, $c_k^\tau$, are bounded by the sum of mass inputs $\sum_{t=1}^\tau x^t+...
user avatar
1 vote
0 answers
39 views

Machine learning with raw data alone / or raw data with its statistics

My question is very general and it does not originate from a specific problem. Let's assume that, through experience, we have learned that some statistical property of a set of data is important in ...
user avatar
7 votes
1 answer
808 views

Is LSTM a subcategory of RNN?

Is the LSTM-Architecture a subcategory of RNNs? Or are they totally different? Literature doesn't seem to be unitary on this. This figure appears to explain the models to be alternatives, but I ...
user avatar
0 votes
1 answer
58 views

Do LSTM in tensorflow work sequentially or in parallel

I have a basic understanding how a cell and a layer of an LSTM works. However, I get confused by what "number of units" (as termed in tensorflow) exactly means. A unit is, as far as I ...
user avatar
0 votes
1 answer
77 views

Reinforcing Learning when action has no effect on the environment

I am trying to get my head around a problem where the action by the agent can not change the environment. Without going into details, my problem is about error correction in an stochastic environment. ...
user avatar
2 votes
1 answer
32 views

What type of neural network do you need if you want to detect an action or dynamic pattern instead of a static pattern?

Let's say that you want to detect if a man is running, walking, or dancing instead of just detecting a man still. What type of neural networks will you use for this purpose?
user avatar
0 votes
0 answers
47 views

What's the best way to feed stories to a neural network?

I'm trying to train a model that would generate stories. I have a dataset of 2000 stories prepared. They are tokenized and one-hot encoded. I can't load them all at once as a one big dataset, because ...
user avatar
1 vote
0 answers
19 views

The results are not correct when predicting the future for a very long period of time with LSTM

I am currently using LSTM to try to predict future data in AirPassengers.csv. This is current code op my Colab (sorry for the comments are Japanese) https://colab.research.google.com/drive/...
user avatar
  • 13
0 votes
0 answers
22 views

Time series forecasting with some challenges

I'm attempting to devise a strategy to make time series forecasts based on costs accumulated over time. My dataset contains about 7500 time-series sequences (call it an instance for now), each having ...
user avatar
0 votes
2 answers
29 views

What is the reason for a training loss that drops but validation that NEVER does

I've been working on learning about NLP via a beginners competition on Kaggle. I first trained a model with an embedding layer and then a simple linear layer. I actually got way better than a flip of ...
user avatar
0 votes
0 answers
38 views

LSTM and CNN - feature engineering and order for time series classification

My questions are related to multivariate time series classification, hence it may differ from forecasting problems. I can have either variable (entire history of the series) or fixed time steps (...
user avatar
  • 1
0 votes
0 answers
28 views

LSTM predictions are one time step lagging

My problem involves electricity prediction (time-series problem) for 1-hour ahead. I am using LSTM to forecast. Length of Dataset: 1 year at one-hour interval Input: Outdoor Temperature (Ot), ...
user avatar
0 votes
0 answers
44 views

How Long Short Term Memory (LSTM) work for time series classification?

I first got the concept of LSTM on how it works word to word prediction etc. However, I want to know how it work with the time-series classification. For example I have the follwing data (see image ...
user avatar
0 votes
0 answers
21 views

How will actual labels be matched with predicted labels when LSTM discards data even from current time stamp input data?

I read the tutorial of LSTM from here. However, I have certain doubts that I need to address. Since we use true labels and do not remove anything from the original data, then how is it possible for ...
user avatar
0 votes
0 answers
32 views

Time series forecasting for multiple objects with common features

I know the title of this question may raise an eyebrow, but I can't find the technical terms to define or investigate the actual problem. To demonstrate my problem with a simple hypothetical scenario: ...
user avatar
0 votes
1 answer
57 views

Using a Neural Network (LSTM) to approve/reject word-type sequences

I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence. For instance: "Mike's Airplane" would output "...
user avatar
0 votes
1 answer
30 views

How can I use a prediction model (e.g., ARMA model or LSTM) for multi-variate data?

I have a question I have had a dataset below ...
user avatar
0 votes
0 answers
18 views

End-to-end learning using LSTM-AE

I want to use prediction models like LSTM-AE to predict time-series data. The feature that the neural network should learn is in frequency between 40-60Hz. So, in order to learn the feature more ...
user avatar
  • 111
1 vote
1 answer
36 views

How can I address missing values for LSTM?

I'm a student and writing my first paper for submission on conference. I have a question there is a dataset below. this is temporal-spatial dataset. ...
user avatar
1 vote
1 answer
71 views

Why doesn't the LSTM model improve the time-series forecasting significantly with respect to the MLP model?

I have recently started learning time series forecasting. I have a dataset of the weekly payment history of 10k clients over 1 year, and I want to predict the future 5 payments for a test set of 1k ...
user avatar
0 votes
0 answers
34 views

Keras MLP performing better than Transformers

I'm working on a comparative study using some models in a sentiment analysis task: MLPs and LSTMs with and without the use of word embeddings (GloVe and Word2Vec) and two Transformer-based models (...
user avatar
0 votes
0 answers
17 views

Feature selection by Simple regression vs finite impulse response (FIR) method (on TIME Series analysis)

We are working on prediction one company production estimation and the main field of works is like stock market prediction(Time series analysis and process data). So I have some comment on using ...
user avatar
1 vote
0 answers
42 views

LSTM Recursive Forecast

I am confused about the way the LSTM networks work when forecasting with a horizon that is not finite, but I'm rather searching for a prediction in whatever time in future. In physical terms, I would ...
user avatar

1
2 3 4 5 6