Questions tagged [long-short-term-memory]
For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".
276
questions
0
votes
1
answer
35
views
Recognize patterns within random sequences
I am familiar with ANNs as I studied them back in the days for regression and currently I'm working with CNN's for image recognition. But recently I was reading more about pattern recognition in ...
- 101
0
votes
0
answers
11
views
LRP (Layer wise relevance propagation ) backward pass for two layer LSTM Networks
I am trying to calculate relevance scores for each time step of the input of my network which is a time series of shape (batch_size = 7000,sequence_length = 20,number of features = 1).
my network has ...
- 1
0
votes
1
answer
21
views
Difference between sequence length and hidden size in LSTM
It does not come clear to me how the seq_length is not the exact same as the hidden_size in LSTMs.
For example, in the next ...
- 1
3
votes
2
answers
586
views
Can LSTM model use ReLU or LeakyReLU as the activation funtion?
Can LSTM model use ReLU or LeakyReLU as the activation funtion? If so, when should one use tanh and when should one use ReLU or LeakyReLU?
- 57
0
votes
1
answer
51
views
What should I do, reinforcement learning agent gives different result on every train?
I'm using PPO+LSTM to create a trading bot. The agent is trained on 3 years of data and tested on 1 year. Every time I train the agent with same set of hyper-parameters, I get very different results ...
- 1
1
vote
1
answer
28
views
Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?
I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
- 111
0
votes
1
answer
26
views
Why final memory state equals to the last hidden state of entire hidden state sequence?
when return_sequences=True and return_state=True, a TensorFlow LSTM outputs the hidden states of the LSTM cell along with the ...
- 123
1
vote
0
answers
12
views
SCINet: how does interactive learning work?
i'm having some trouble understanding how does the basic building block of a SCINet works. In the paper the author describes the SCI-block with the following figure:
In which $\phi$, $\theta$, $\eta$ ...
0
votes
0
answers
20
views
How can I transform the LSTM output to an embedding matrix of actions?
Page 3 of the paper Feudal Networks for Hierarchical Reinforcement Learning describes producing an 'embedding matrix' $U$ of size $(|a| \times k)$ from the output of an LSTM, given a $(d \times 1)$ ...
- 1
0
votes
0
answers
15
views
Text Classification Model unable to learn
I am trying to build a text classification model. When I train the model it is unable to improve accuracy and at some point accuracy even decreases and loss increases.
I have researched for possible ...
- 1
0
votes
1
answer
47
views
How to identify overfitting in LSTM-RNN using metrics?
How can I identify overfitting on a RNN-LSTM with the following metrics: RMSE, MSE, RAE, R-squared ? I have searched papers and google results. I don't see something clear to my mind. Also I rarely ...
- 105
0
votes
0
answers
11
views
Modeling the previous inputs to affect next output in Machine learning
I am working on a dataset contains one output variable and a number of input variables.The data looks like the following:
Y, X1, X2, X3, X4
7, 5, 0.7, 8, 9
3, 6, 0.3, 9, 9
....
Where Y is the output ...
0
votes
0
answers
21
views
LSTM with time-series data transform
I have a time series data which has distinct time steps. For example, one time series data was recorded with the 1/30(min) time step, but some other data may have 1/15(min), 1/6(min), 1/5(min) time ...
- 101
0
votes
0
answers
21
views
bad prediction when having noise on the data: LSTM time-series regression
I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
1
vote
0
answers
12
views
Where exactly is permutation happening in equation 5 of the paper "Learning with Sets in Multiple Instance Regression Applied to Remote Sensing"?
I am reading the article Learning with Sets in Multiple Instance Regression Applied to Remote Sensing about creating an embedding which is order-invariant to inputs ($m_{l}$). They referred to order-...
- 23
0
votes
0
answers
21
views
Generalize and optimize a model for multiple time series
I have a physics equation that takes so much time to be solved computationally.
So the idea is to optimize this computational time with Machine Learning techniques. I've already generated data ...
- 21
0
votes
0
answers
46
views
How to evaluate the embeddings of a model?
If you have a task of extracting embeddings from a model (such as penultimate layer - pre-last layer of the model), would you train the model on a benchmark similar dataset (if there were) or train on ...
0
votes
0
answers
17
views
When training an LSTM, should you pad your dataset so the sequence length is static, or should it be variable?
I am putting together an LSTM network using visual basic. It's more of a learning exercise really, but it's also the only programming language I have access too at work.
I am unsure of how to prepare ...
- 323
2
votes
2
answers
390
views
Val loss doesn’t decrease after a certain number of epochs
I’m working on a classification problem (500 classes). My NN has 3 fully connected layers, followed by an LSTM layer. I use nn.CrossEntropyLoss() as my loss ...
- 65
0
votes
0
answers
8
views
Multivariate RNN with different sequence length for each feature
I want to use an LSTM to predict a stock price. I have a few groups of features that I am willing to use. Some of these features are fundamentals like earnings and sales and others are technicals like ...
0
votes
1
answer
94
views
Extremely good results in RNN-LSTM python code!! How can this happen?
I am using this code here: https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/ and more specifically the python code under the (1st) paragraph "...
- 105
1
vote
2
answers
73
views
How does Seq2Seq with attention actually use the attention (i.e. the context vector)?
For neural machine translation, there's this model "Seq2Seq with attention", also known as the "Bahdanau architecture" (a good image can be found on this page), where instead of ...
- 161
0
votes
0
answers
18
views
The dimensions of State-value function V and Q-value in reinforcement learning when using LSTM in the Critic network
I'm dealing with a project where the state s is sequential data with a fixed length. In this case, both the actor and critic have to contain LSTM.
For the critic, I didn't flatten the intermediate ...
0
votes
0
answers
138
views
Do Transformers and LSTMs use the same word embeddings (except for the position encoding, which only Transformers use)?
In NLP, the first step is always to "convert" the given words of a sentence into representation vectors (word embeddings).
As I understand it, in the case of transformers, the words/...
- 11
0
votes
0
answers
84
views
Why remove stop words, numbers in a conversational chatbot?
I have been working on a conversational chatbot recently using movie dialogues corpus dataset, since i am very new to this i started to see if there's already code available for chatbots. I came ...
0
votes
0
answers
11
views
Predicting time spent to build a metal piece using RNN
My data consists in many metal pieces which are put together to make a final metal mould.
To make each of this metal pieces, machinery recieves many operations to follow, like chopping, facing, etc... ...
- 1
0
votes
1
answer
54
views
Does sequence matter in LSTM?
In general, as long as the items are in order as per time sequence, does it matter if it's in ascending time sequence or descending time sequence for LSTM within the input vector?
eg:
ABCDEF -> G
...
0
votes
0
answers
20
views
How to Decide on the Structure of a Neural Network for Time Series Forecasting?
Apologies for the noob question. I am attempting time series forecasting (with a combination of lag and categorical features) using tensorflow and struggling to find an optimal combination of RNN/LSTM ...
- 111
0
votes
0
answers
16
views
Selecting neural network architecture for battery degradation problem
I want to design a neural network capable of detecting the degradation of the capacity of a battery. When the capacity is ok, the battery performs like this (x axis is the time in seconds and y axis ...
- 101
0
votes
1
answer
80
views
Choice of LSTM for price prediction
I have a dataset with features (f) for different stocks (S) and want to infer for price using an LSTM model. Here is my df:
year
S1_price
S1_f1
S1_f2
S2_price
S2_f1
S2_f2
Sn_price
Sn_f1
Sn_f2
2010
...
- 21
0
votes
1
answer
31
views
What is the training accuracy of this model?
I’m trying to classifiy ECG signals using LSTM and MATLAB, the above plot shows that the training accuracy of the system is 100% but when I apply this code to calculate and get the accuracy I get only ...
0
votes
1
answer
45
views
Limitations of LSTMs
I'm training an LSTM model for classification on accelerometer data, and I get better results when I downsample the signal to 25 Hz than when I use a 50 Hz signal.
I use the same time frame of 1.5 ...
- 111
0
votes
0
answers
56
views
How is it possible to use batches of data from within the same sequence with an LSTM?
ETA: More concise wording: Why do some implementations use batches of data taken from within the same sequence? Does this not make the cell state useless?
Using the example of an LSTM, it has a hidden ...
- 1,316
2
votes
2
answers
105
views
LSTM exploding? - multiple parallel time series with multiple variables
I have the following situation:
Stock
Time_Stamps
Feature_1
Feature_2
Feature_n
Price
Stock_1
2019
0.5
1.0
1.0
100
Stock_1
2020
0.7
1.3
0.9
90
Stock_2
2019
0.3
0.9
1.1
110
Stock_2
2020
0.2
0.8
1....
- 21
1
vote
1
answer
300
views
Where is memory stored in a chatbot like LaMDA?
I have a basic understanding of how neural networks work, and I have always thought that those chatbots work in a similar way (but I might be wrong): they take an input, shape it in a way that can be ...
- 113
1
vote
1
answer
34
views
How to identify important features in data?
I have a couple opportunities to write a paper, or papers over some of the neural networks I have made.
I was wondering if there are anyways to figure out why the neural network classifies the data I ...
- 11
1
vote
1
answer
436
views
Does LSTM provide any unique value or advantages compared to other algorithms, including "vanilla" RNN?
I have heard a lot of hype around LSTM for all kinds of time-series based applications including NLP. Despite this, I haven't seen many (if any) applications of LSTM where LSTM performs uniquely well ...
- 312
0
votes
0
answers
48
views
Finding "look_back" & "look_ahead" hyper-parameters for Seq2Seq models
For Seq2Seq deep learning architectures, viz., LSTM/GRU and multivariate, multistep time series forecasting, it is important to convert the data to a 3D dimension: (batch_size, look_back, ...
- 215
1
vote
2
answers
492
views
What is the advantage of adding CNN to LSTM for forecasting sequential data?
I am working with simulated sequential data and the goal is to forecast that data. Long-short-term-memory (LSTM) is one of the most advanced models to forecast time series according to this post. I ...
- 155
-1
votes
1
answer
301
views
How Many Hidden Units in an LSTM? [duplicate]
Is there any rule of thumb for choosing the number of hidden units in an LSTM? Is it similar to hidden neurons in a regular feedforward neural network? I'm getting better results with my LSTM when I ...
- 11
0
votes
1
answer
997
views
What is the difference between CNN-LSTM and RNN?
I'm starting to study RNN for a project of video prediction, but I encounter these CNN-LSTM models. Initially, I thought that is another name for RNN, but I think I get it wrong. Since I'm a beginner ...
0
votes
1
answer
428
views
Do the values over 0.5 mean my model classified the data as a "1" label and vice versa?
I am doing binary classification using an LSTM and my output layer is 1 neuron with a sigmoid function. My labels are either 0 or 1.
...
- 3
0
votes
1
answer
405
views
Training strategy on continuous video stream with CNN-LSTM
I have videos that are each about 30-40 mins long. With the first 5-10 mins (at 60fps, can be down-sampled to 5fps) are one type of activity that would be categorized by label-1 and the rest of the ...
- 101
1
vote
1
answer
801
views
Sequence Embedding using embedding layer: how does the network architecture influence it?
I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there ...
0
votes
1
answer
33
views
time series analysis: predict number and type of service
I have temporal data regarding the number of customers who requested a specific service in a given period (month and year). Below is a small excerpt from the dataset:
Month-year: month and year when ...
- 65
0
votes
0
answers
75
views
LSTM - What kind of data should contain every dimension of input LSTM matrix, where does specific dimension points to?
I am a beginner and I have a hard time understanding inputs and outputs of LSTM.
So from the begining, I am trying to create multivariate input&output LSTM for time series forecasting.
Thankfully ...
0
votes
1
answer
109
views
Is my intuition about RNN wrong?
Until today, my intuition about RNN (LSTM/GRU) was that this is some kind of NN that can remember previous inputs.
Consider a task where you need to predict 0 if the previous input was 1. For example: ...
- 109
1
vote
1
answer
2k
views
What is the time complexity for testing a stacked LSTM model?
In the data preparation phase, we have to divide the dataset into two parts: the training dataset and the test dataset.
I have seen this post regarding the time complexity for training a model.
...
- 113
-1
votes
1
answer
59
views
Is my dataset a time series dataset? and should I use an LSTM?
I have a dataset where I am recording temperature after every 4milliseconds till 500 and another feature "conductivity value". The length of the dataset is around a 1000 rows. I need to find ...
- 51
1
vote
1
answer
368
views
Is a recurrent layer same as LSTM or single-layered LSTM?
In MLP, there are neurons that form a layer. Each hidden layer gives a vector of number that is the output of that layer.
In CNN, there are kernels that form a convolutional layer. Each layer gives ...
- 3,501