Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

Filter by
Sorted by
Tagged with
0 votes
1 answer
35 views

Recognize patterns within random sequences

I am familiar with ANNs as I studied them back in the days for regression and currently I'm working with CNN's for image recognition. But recently I was reading more about pattern recognition in ...
0 votes
0 answers
11 views

LRP (Layer wise relevance propagation ) backward pass for two layer LSTM Networks

I am trying to calculate relevance scores for each time step of the input of my network which is a time series of shape (batch_size = 7000,sequence_length = 20,number of features = 1). my network has ...
  • 1
0 votes
1 answer
21 views

Difference between sequence length and hidden size in LSTM

It does not come clear to me how the seq_length is not the exact same as the hidden_size in LSTMs. For example, in the next ...
3 votes
2 answers
586 views

Can LSTM model use ReLU or LeakyReLU as the activation funtion?

Can LSTM model use ReLU or LeakyReLU as the activation funtion? If so, when should one use tanh and when should one use ReLU or LeakyReLU?
0 votes
1 answer
51 views

What should I do, reinforcement learning agent gives different result on every train?

I'm using PPO+LSTM to create a trading bot. The agent is trained on 3 years of data and tested on 1 year. Every time I train the agent with same set of hyper-parameters, I get very different results ...
1 vote
1 answer
28 views

Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?

I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
0 votes
1 answer
26 views

Why final memory state equals to the last hidden state of entire hidden state sequence?

when return_sequences=True and return_state=True, a TensorFlow LSTM outputs the hidden states of the LSTM cell along with the ...
  • 123
1 vote
0 answers
12 views

SCINet: how does interactive learning work?

i'm having some trouble understanding how does the basic building block of a SCINet works. In the paper the author describes the SCI-block with the following figure: In which $\phi$, $\theta$, $\eta$ ...
0 votes
0 answers
20 views

How can I transform the LSTM output to an embedding matrix of actions?

Page 3 of the paper Feudal Networks for Hierarchical Reinforcement Learning describes producing an 'embedding matrix' $U$ of size $(|a| \times k)$ from the output of an LSTM, given a $(d \times 1)$ ...
  • 1
0 votes
0 answers
15 views

Text Classification Model unable to learn

I am trying to build a text classification model. When I train the model it is unable to improve accuracy and at some point accuracy even decreases and loss increases. I have researched for possible ...
0 votes
1 answer
47 views

How to identify overfitting in LSTM-RNN using metrics?

How can I identify overfitting on a RNN-LSTM with the following metrics: RMSE, MSE, RAE, R-squared ? I have searched papers and google results. I don't see something clear to my mind. Also I rarely ...
0 votes
0 answers
11 views

Modeling the previous inputs to affect next output in Machine learning

I am working on a dataset contains one output variable and a number of input variables.The data looks like the following: Y, X1, X2, X3, X4 7, 5, 0.7, 8, 9 3, 6, 0.3, 9, 9 .... Where Y is the output ...
0 votes
0 answers
21 views

LSTM with time-series data transform

I have a time series data which has distinct time steps. For example, one time series data was recorded with the 1/30(min) time step, but some other data may have 1/15(min), 1/6(min), 1/5(min) time ...
0 votes
0 answers
21 views

bad prediction when having noise on the data: LSTM time-series regression

I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
1 vote
0 answers
12 views

Where exactly is permutation happening in equation 5 of the paper "Learning with Sets in Multiple Instance Regression Applied to Remote Sensing"?

I am reading the article Learning with Sets in Multiple Instance Regression Applied to Remote Sensing about creating an embedding which is order-invariant to inputs ($m_{l}$). They referred to order-...
  • 23
0 votes
0 answers
21 views

Generalize and optimize a model for multiple time series

I have a physics equation that takes so much time to be solved computationally. So the idea is to optimize this computational time with Machine Learning techniques. I've already generated data ...
0 votes
0 answers
46 views

How to evaluate the embeddings of a model?

If you have a task of extracting embeddings from a model (such as penultimate layer - pre-last layer of the model), would you train the model on a benchmark similar dataset (if there were) or train on ...
0 votes
0 answers
17 views

When training an LSTM, should you pad your dataset so the sequence length is static, or should it be variable?

I am putting together an LSTM network using visual basic. It's more of a learning exercise really, but it's also the only programming language I have access too at work. I am unsure of how to prepare ...
  • 323
2 votes
2 answers
390 views

Val loss doesn’t decrease after a certain number of epochs

I’m working on a classification problem (500 classes). My NN has 3 fully connected layers, followed by an LSTM layer. I use nn.CrossEntropyLoss() as my loss ...
0 votes
0 answers
8 views

Multivariate RNN with different sequence length for each feature

I want to use an LSTM to predict a stock price. I have a few groups of features that I am willing to use. Some of these features are fundamentals like earnings and sales and others are technicals like ...
0 votes
1 answer
94 views

Extremely good results in RNN-LSTM python code!! How can this happen?

I am using this code here: https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/ and more specifically the python code under the (1st) paragraph "...
1 vote
2 answers
73 views

How does Seq2Seq with attention actually use the attention (i.e. the context vector)?

For neural machine translation, there's this model "Seq2Seq with attention", also known as the "Bahdanau architecture" (a good image can be found on this page), where instead of ...
  • 161
0 votes
0 answers
18 views

The dimensions of State-value function V and Q-value in reinforcement learning when using LSTM in the Critic network

I'm dealing with a project where the state s is sequential data with a fixed length. In this case, both the actor and critic have to contain LSTM. For the critic, I didn't flatten the intermediate ...
0 votes
0 answers
138 views

Do Transformers and LSTMs use the same word embeddings (except for the position encoding, which only Transformers use)?

In NLP, the first step is always to "convert" the given words of a sentence into representation vectors (word embeddings). As I understand it, in the case of transformers, the words/...
  • 11
0 votes
0 answers
84 views

Why remove stop words, numbers in a conversational chatbot?

I have been working on a conversational chatbot recently using movie dialogues corpus dataset, since i am very new to this i started to see if there's already code available for chatbots. I came ...
0 votes
0 answers
11 views

Predicting time spent to build a metal piece using RNN

My data consists in many metal pieces which are put together to make a final metal mould. To make each of this metal pieces, machinery recieves many operations to follow, like chopping, facing, etc... ...
  • 1
0 votes
1 answer
54 views

Does sequence matter in LSTM?

In general, as long as the items are in order as per time sequence, does it matter if it's in ascending time sequence or descending time sequence for LSTM within the input vector? eg: ABCDEF -> G ...
0 votes
0 answers
20 views

How to Decide on the Structure of a Neural Network for Time Series Forecasting?

Apologies for the noob question. I am attempting time series forecasting (with a combination of lag and categorical features) using tensorflow and struggling to find an optimal combination of RNN/LSTM ...
  • 111
0 votes
0 answers
16 views

Selecting neural network architecture for battery degradation problem

I want to design a neural network capable of detecting the degradation of the capacity of a battery. When the capacity is ok, the battery performs like this (x axis is the time in seconds and y axis ...
  • 101
0 votes
1 answer
80 views

Choice of LSTM for price prediction

I have a dataset with features (f) for different stocks (S) and want to infer for price using an LSTM model. Here is my df: year S1_price S1_f1 S1_f2 S2_price S2_f1 S2_f2 Sn_price Sn_f1 Sn_f2 2010 ...
0 votes
1 answer
31 views

What is the training accuracy of this model?

I’m trying to classifiy ECG signals using LSTM and MATLAB, the above plot shows that the training accuracy of the system is 100% but when I apply this code to calculate and get the accuracy I get only ...
0 votes
1 answer
45 views

Limitations of LSTMs

I'm training an LSTM model for classification on accelerometer data, and I get better results when I downsample the signal to 25 Hz than when I use a 50 Hz signal. I use the same time frame of 1.5 ...
  • 111
0 votes
0 answers
56 views

How is it possible to use batches of data from within the same sequence with an LSTM?

ETA: More concise wording: Why do some implementations use batches of data taken from within the same sequence? Does this not make the cell state useless? Using the example of an LSTM, it has a hidden ...
  • 1,316
2 votes
2 answers
105 views

LSTM exploding? - multiple parallel time series with multiple variables

I have the following situation: Stock Time_Stamps Feature_1 Feature_2 Feature_n Price Stock_1 2019 0.5 1.0 1.0 100 Stock_1 2020 0.7 1.3 0.9 90 Stock_2 2019 0.3 0.9 1.1 110 Stock_2 2020 0.2 0.8 1....
1 vote
1 answer
300 views

Where is memory stored in a chatbot like LaMDA?

I have a basic understanding of how neural networks work, and I have always thought that those chatbots work in a similar way (but I might be wrong): they take an input, shape it in a way that can be ...
1 vote
1 answer
34 views

How to identify important features in data?

I have a couple opportunities to write a paper, or papers over some of the neural networks I have made. I was wondering if there are anyways to figure out why the neural network classifies the data I ...
1 vote
1 answer
436 views

Does LSTM provide any unique value or advantages compared to other algorithms, including "vanilla" RNN?

I have heard a lot of hype around LSTM for all kinds of time-series based applications including NLP. Despite this, I haven't seen many (if any) applications of LSTM where LSTM performs uniquely well ...
0 votes
0 answers
48 views

Finding "look_back" & "look_ahead" hyper-parameters for Seq2Seq models

For Seq2Seq deep learning architectures, viz., LSTM/GRU and multivariate, multistep time series forecasting, it is important to convert the data to a 3D dimension: (batch_size, look_back, ...
  • 215
1 vote
2 answers
492 views

What is the advantage of adding CNN to LSTM for forecasting sequential data?

I am working with simulated sequential data and the goal is to forecast that data. Long-short-term-memory (LSTM) is one of the most advanced models to forecast time series according to this post. I ...
  • 155
-1 votes
1 answer
301 views

How Many Hidden Units in an LSTM? [duplicate]

Is there any rule of thumb for choosing the number of hidden units in an LSTM? Is it similar to hidden neurons in a regular feedforward neural network? I'm getting better results with my LSTM when I ...
  • 11
0 votes
1 answer
997 views

What is the difference between CNN-LSTM and RNN?

I'm starting to study RNN for a project of video prediction, but I encounter these CNN-LSTM models. Initially, I thought that is another name for RNN, but I think I get it wrong. Since I'm a beginner ...
0 votes
1 answer
428 views

Do the values over 0.5 mean my model classified the data as a "1" label and vice versa?

I am doing binary classification using an LSTM and my output layer is 1 neuron with a sigmoid function. My labels are either 0 or 1. ...
0 votes
1 answer
405 views

Training strategy on continuous video stream with CNN-LSTM

I have videos that are each about 30-40 mins long. With the first 5-10 mins (at 60fps, can be down-sampled to 5fps) are one type of activity that would be categorized by label-1 and the rest of the ...
  • 101
1 vote
1 answer
801 views

Sequence Embedding using embedding layer: how does the network architecture influence it?

I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there ...
0 votes
1 answer
33 views

time series analysis: predict number and type of service

I have temporal data regarding the number of customers who requested a specific service in a given period (month and year). Below is a small excerpt from the dataset: Month-year: month and year when ...
  • 65
0 votes
0 answers
75 views

LSTM - What kind of data should contain every dimension of input LSTM matrix, where does specific dimension points to?

I am a beginner and I have a hard time understanding inputs and outputs of LSTM. So from the begining, I am trying to create multivariate input&output LSTM for time series forecasting. Thankfully ...
0 votes
1 answer
109 views

Is my intuition about RNN wrong?

Until today, my intuition about RNN (LSTM/GRU) was that this is some kind of NN that can remember previous inputs. Consider a task where you need to predict 0 if the previous input was 1. For example: ...
  • 109
1 vote
1 answer
2k views

What is the time complexity for testing a stacked LSTM model?

In the data preparation phase, we have to divide the dataset into two parts: the training dataset and the test dataset. I have seen this post regarding the time complexity for training a model. ...
-1 votes
1 answer
59 views

Is my dataset a time series dataset? and should I use an LSTM?

I have a dataset where I am recording temperature after every 4milliseconds till 500 and another feature "conductivity value". The length of the dataset is around a 1000 rows. I need to find ...
1 vote
1 answer
368 views

Is a recurrent layer same as LSTM or single-layered LSTM?

In MLP, there are neurons that form a layer. Each hidden layer gives a vector of number that is the output of that layer. In CNN, there are kernels that form a convolutional layer. Each layer gives ...
  • 3,501

1
2 3 4 5 6