Skip to main content

Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

What is the exact purpose of input modulation gate in LSTMs?

Basically, I was learning about LSTMs where I found LSTMs are made up of three gates: The forget gate, input gate and output gate. However, I came across some sources that state there is a fourth gate ...
MrIzzat's user avatar
0 votes
0 answers
21 views

Distribution function for lstm and MLP

I have total of 6300 samples, 5800 of which are training data, and 500 of which are testing data. We compare the performance of LSTM and multilayer perceptron (MLP) with one hidden layer in terms of ...
HDD's user avatar
  • 101
0 votes
1 answer
20 views

Spatial vs spatiotemporal methods for object counting in low frame-rate videos

I'm currently working on an object counting/density estimation task using low frame rate video (~2 fps) in a traffic setting. I've explored a lot of literature on both spatial methods (i.e. using only ...
yuki's user avatar
  • 11
0 votes
0 answers
62 views

Any reasons LSTM does not pick up any patterns?

I'm trying to teach an LSTM to predict the next values in 3 related series. (Financial data) Unfortunately, it looks like I made some basic mistake and this network never gets past just returning ...
viraptor's user avatar
  • 111
0 votes
0 answers
22 views

Preparation of multivariate time series data

I am doing a university project on index/stock price prediction. I plan to use a combined cnn-lstm model, and I have several different types of data: Open High Low Close Volume, values, fundamental ...
Ivan's user avatar
  • 1
0 votes
0 answers
8 views

Is it possible to apply transfer learning between Temporal Fusion Transformer and sequential architecture LSTM and GRU

If TFT is a pretrained model, is it possible to transfer the weights to sequential neural network models like LSTM,BILSTM and GRU.
Santhana Lakshmi's user avatar
0 votes
1 answer
85 views

xLSTM parallel computation - mismatch in dimensions

In this recent paper, a new architecture is proposed, called xLSTM. I've implemented the sequential version in PyTorch, but it's slower than I would like, so I'm now implementing the parallel version ...
Quaere Verum's user avatar
0 votes
1 answer
29 views

Can I scale subsets of my dataset independently to handle different feature ranges?

I am currently making an LSTM model to predict the change in value of a stock for each day. My dataset consists of 4 years of data for 30 different stocks along with some financial metrics such as RSI,...
Daniel goldney's user avatar
0 votes
0 answers
59 views

Drum sound classification using RNN issues - help needed

I am new to the field of machine learning, even tho I have solid background in semi-related fields (am control system engineer by trade) and as a hobby project I wanted to work a bit with sound ...
APasagic's user avatar
1 vote
2 answers
74 views

Wouldn't residual connections in RNNs solve the vanishing/exploding gradient problem?

I was recently brushing up on my deep-learning basics and came back to RNNs. LSTMs/GRUs and the Transformer architecture were invented to solve RNN's vanishing/exploding gradient problem. I was at ...
Vladislav Korecký's user avatar
0 votes
0 answers
32 views

What is the input size and sequence length for lstm when the input is audio data where each audio file is sampled at the rate of 3000hz?

I have music data which contains raw vocals of some indian classical and devotional songs and I have segmented each file into 20 seconds files. I have used librosa.load function with a sample rate of ...
Sai Abhishek Bhyri's user avatar
0 votes
0 answers
28 views

"The single scalar stored by an LSTM or GRU memory cell" - Deep learning book

I am reading Deep Learning by Goodfellow, Bengio, and Courville, and on page 413, they discuss how to store information using a framework such as a neural Turing machine. Quote: Neural networks excel ...
Kaira's user avatar
  • 101
1 vote
0 answers
284 views

Glass Degradation Video Prediction

We are working on the subject above, where a sequence of $n$ glass frames forms an example with an associate target that is a video of the glass future state. We would like to know if there exists an ...
Filippo Portera's user avatar
0 votes
0 answers
33 views

How can I process financial stock data before passing it to an LSTM for time series classification?

I'm trying to make an LSTM that can classify the next day of a stock as either 1 or 0 for going up or down. The issue I've been having is that Keras tuner seems to stay at a constant value of val_loss ...
JhonDenver's user avatar
0 votes
0 answers
18 views

Extracting features from multiple curves

I am building a model that predicts the SOH of a lithium ion battery. My data are from 600 battery charge cycles as follows: for each cycle I have 3 curves each of length 128 samples: voltage, current ...
deckard1992's user avatar
0 votes
1 answer
72 views

How LSTM really decide what to forget and not?

Currently, I am learning about LSTM, and I understand the intuition behind it, such as how forget gate works (sigmoid function yields a value between 0 and 1; if it is 0 it "completely" ...
Ashraf's user avatar
  • 3
0 votes
0 answers
25 views

Realtime Inference of Stateful Variable-Length Time Series LSTM - Getting around Sequence Length vs Inference Frequency Tradeoff

I am trying to train a neural net for low-latency signal filtering; I've been recommended to use a stateful LSTM architecture for this task, however, having corrected some , it seems to me that trying ...
stellarpower's user avatar
0 votes
0 answers
64 views

Are there guidelines or rules of thumb on how to stack hidden layers in a RNN?

I’m currently working on the prediction of chaotic data and I have decided to see how well would an RNN, namely an LSTM, would do. I am fairly new to the topic of Neural Networks, but I have found a ...
Jxson99's user avatar
  • 103
1 vote
1 answer
1k views

When are Transformers better than LSTMs in time-series tasks such as classification?

I’m working on a time-series classification problem and trying to decide whether to use a Transformer or an LSTM. From what I’ve learned, Transformers are better suited for capturing long-range ...
Mark Cortejo's user avatar
1 vote
1 answer
33 views

Can I do incremental learning with different loss function in neural networks?

I have a saved tensorflow neural network model. I was wondering if it's possible to incrementally train the model but with different nt loss function.
SUNITA GUPTA's user avatar
0 votes
0 answers
12 views

Improving forecast for LSTM with additional data

There are two timeseries X and Y. The timeseries X spans the duration [1 Jan, 20 Jan ] and the Y spans [1 Jan, 25 Jan].  I am interested in timeseries forecasting of variable X for the duration [21 ...
pkj's user avatar
  • 101
0 votes
0 answers
28 views

Simultaneous forecasting and classification

I'm working on a project where I need to perform both forecasting (regression) and classification using time series data. The dataset is labelled. I've been exploring LSTM networks due to their ...
mike7's user avatar
  • 1
0 votes
0 answers
14 views

How can a RNN with 256 cells accept a input of any size?

I built a 3 layered RNN model with 256 cells each using torch. Input feature size is set to 40. Below give a basic Idea on the model. ...
D Star Let's Explore's user avatar
0 votes
0 answers
19 views

How is the accuracy as a metric in a Keras machine learning model calculated? Is it a valuable metric for LSTM

I'm training a LSTM neural network for time series prediction in Keras. During the training of the model, the loss (mse) gradually decreases each epoch, but the accuracy as well as the validation ...
Mappy's user avatar
  • 1
0 votes
0 answers
12 views

Possible Reasons for the Discrepancy in Trainable Parameters of the Extended DeepConvLSTM Model

I have implemented DeepConvLSTM baseline Model input are 60×d frames each representing 60 samples with d features. Frames are fed into four consecutive convolution layers with standard rectified ...
Nafees Ahmed's user avatar
1 vote
1 answer
33 views

LSTM with multiple data streams

I am working on the following problem: I have ~10 weather stations in somewhat approximate areas, at some points during the day (different for each station), I get readings of various data points (...
PenguinHook's user avatar
0 votes
1 answer
35 views

Sequential models and distribution shift in RL

We know the problem of "distribution shift" in deep Reinforcement Learning, where the change in policy during training affects the behavior of the agent and therefore the distribution of the ...
SuperTardigrade's user avatar
0 votes
0 answers
23 views

ROC curve for multiclassification - results sound not correct

I'm working on a multiclassification task using LSTM algorithm, i generated my roc curve plots but they give scores like 1 , 0.99, 0.97 however i have an accuracy of 0.97, Precision 0.65, Sensitivity/...
biihu's user avatar
  • 1
0 votes
0 answers
45 views

How to setup correctly a sequence generation task with RL/policy gradient learning?

I've a pretrained model for sequence generation that I'd like to improve with RL but there are several shady points. So, I have the following model and loss function: ...
eris's user avatar
  • 1
0 votes
0 answers
42 views

Updating custom output layers of an LSTM network

I have a text generation task learning to predict the next word with an LSTM network with multiple output layers. After the generation of a sentence has finished, I calculate a reward for the whole ...
eris's user avatar
  • 1
0 votes
1 answer
29 views

Seq2Seq model- Confusing about the dimension of Seq2Seq model [closed]

I am new to Seq2Seq and hope to find a proper guildances, advices. I am doing a Project from an online course so I can not give the material but I got my Project notebook on Github I want to ask ...
QH.Chu's user avatar
  • 3
0 votes
0 answers
21 views

How do I find a similar RNN as a starting point?

I am new to machine learning and neural networks and I want to create a neural network for a study project. I would like to create a RNN, that uses one (A) or several time series (with the same length,...
xlaub's user avatar
  • 1
0 votes
0 answers
52 views

What are the *non-cost-related* reasons RNN+Attention underperform Transformers?

There are obvious trainability and performance challenges with RNNs, such as having to process in serial and BPTT. But let's say we magically had an "optimal" set of weights for the RNN + ...
llllvvuu's user avatar
  • 146
1 vote
1 answer
118 views

How does the linear layer step work and what should I do at the end of the LSTM? [closed]

So basically I've read some text about LSTM, and luckily they mentioned the linear layer step at the end of the LSTM Process. However, they didn't explain how it works or what I would need to convert ...
Anish Kommireddy's user avatar
0 votes
1 answer
67 views

In theory, LSTM should be adequate to solve my problem, but it doesn't

I am currently doing my Master's Thesis in the area of wireless sensing using radio waves. I have 4 receiver distributed in the corners of my room and each of them records so called Channel Impulse ...
binaryBigInt's user avatar
0 votes
0 answers
74 views

LSTM Ensemble: Combine low, mid, and high frequency time series data

I am trying to implement time series classification, but I am struggling a bit with the fact that my multivariate data has mixed frequencies. I have about 10 variables that are updated every minute, ...
Ai4l2s's user avatar
  • 121
0 votes
2 answers
618 views

LUT-Based Sigmoid and Tanh Activation-Functions in Integer Quantized Networks

I want to understand how activation functions, specifically tanh and sigmoid, are used in int8 quantized neural networks. Even more specific, I want to understand a Look-up-Table based approach. My ...
Necrotos's user avatar
0 votes
0 answers
11 views

Can I combined the trained model between different source but same model structure?

Here I got two different deep-learning models that were trained by LSTM and time-series data. The data is the usage percentage of CPU from two different computers. Each computer job was the same. It ...
orde.r's user avatar
  • 103
0 votes
1 answer
2k views

Which preprocessing is the correct way to forecast time-series data using LSTM?

I just started to study time-series forecasting using RNN. I have a few months of time series data that was an hour unit. The data is a kind of percentage value of my little experiment and I would ...
orde.r's user avatar
  • 103
1 vote
0 answers
45 views

The model's accuracy becomes suddenly so unreasonably good at beginning of the training process. I need an explaination

I am practicing machine translation using seq2seq model (more specifically with GRU/LSTM units). The following is my first model: This model first archived about 0.03 accuracy score and gradually ...
Đạt Trần's user avatar
0 votes
1 answer
25 views

Regression Model diverging after adding a new feature with higher variance and magnitude

In a time series regression problem I'm predicting "change" rather than the actual intended value i.e Instead of: ...
Darren Rahnemoon's user avatar
1 vote
1 answer
78 views

Transfer Learning for Solar Energy Production Forecasting with LSTM: Generalized vs. Specialized Models

I am working on a solar energy production forecasting problem using LSTM multi-step models to predict 1/4/8h ahead of solar energy production for different solar installations. Our goal is to help ...
Guilherme Vieira's user avatar
1 vote
1 answer
159 views

Recognize patterns within random sequences

I am familiar with ANNs as I studied them back in the days for regression and currently I'm working with CNN's for image recognition. But recently I was reading more about pattern recognition in ...
FELIPE_RIBAS's user avatar
0 votes
1 answer
2k views

Difference between sequence length and hidden size in LSTM

It does not come clear to me how the seq_length is not the exact same as the hidden_size in LSTMs. For example, in the next ...
moth123's user avatar
  • 31
3 votes
2 answers
2k views

Can LSTM model use ReLU or LeakyReLU as the activation funtion?

Can LSTM model use ReLU or LeakyReLU as the activation funtion? If so, when should one use tanh and when should one use ReLU or LeakyReLU?
BlueSnake's user avatar
1 vote
2 answers
234 views

What should I do, reinforcement learning agent gives different result on every train?

I'm using PPO+LSTM to create a trading bot. The agent is trained on 3 years of data and tested on 1 year. Every time I train the agent with same set of hyper-parameters, I get very different results ...
ad124j2's user avatar
  • 11
1 vote
1 answer
47 views

Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?

I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
Krusty the Clown's user avatar
0 votes
1 answer
125 views

Why final memory state equals to the last hidden state of entire hidden state sequence?

when return_sequences=True and return_state=True, a TensorFlow LSTM outputs the hidden states of the LSTM cell along with the ...
noone's user avatar
  • 123
1 vote
0 answers
23 views

SCINet: how does interactive learning work?

i'm having some trouble understanding how does the basic building block of a SCINet works. In the paper the author describes the SCI-block with the following figure: In which $\phi$, $\theta$, $\eta$ ...
Juan Hirschmann's user avatar
0 votes
1 answer
81 views

How to identify overfitting in LSTM-RNN using metrics?

How can I identify overfitting on a RNN-LSTM with the following metrics: RMSE, MSE, RAE, R-squared ? I have searched papers and google results. I don't see something clear to my mind. Also I rarely ...
jim's user avatar
  • 105

1
2 3 4 5
7