Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

Filter by
Sorted by
Tagged with
1 vote
1 answer
16 views

Can I do incremental learning with different loss function in neural networks?

I have a saved tensorflow neural network model. I was wondering if it's possible to incrementally train the model but with different nt loss function.
SUNITA GUPTA's user avatar
0 votes
0 answers
12 views

Improving forecast for LSTM with additional data

There are two timeseries X and Y. The timeseries X spans the duration [1 Jan, 20 Jan ] and the Y spans [1 Jan, 25 Jan].  I am interested in timeseries forecasting of variable X for the duration [21 ...
pkj's user avatar
  • 101
0 votes
0 answers
25 views

Simultaneous forecasting and classification

I'm working on a project where I need to perform both forecasting (regression) and classification using time series data. The dataset is labelled. I've been exploring LSTM networks due to their ...
mike7's user avatar
  • 1
0 votes
0 answers
13 views

How can a RNN with 256 cells accept a input of any size?

I built a 3 layered RNN model with 256 cells each using torch. Input feature size is set to 40. Below give a basic Idea on the model. ...
D Star Let's Explore's user avatar
0 votes
0 answers
12 views

How is the accuracy as a metric in a Keras machine learning model calculated? Is it a valuable metric for LSTM

I'm training a LSTM neural network for time series prediction in Keras. During the training of the model, the loss (mse) gradually decreases each epoch, but the accuracy as well as the validation ...
Mappy's user avatar
  • 1
0 votes
0 answers
12 views

Possible Reasons for the Discrepancy in Trainable Parameters of the Extended DeepConvLSTM Model

I have implemented DeepConvLSTM baseline Model input are 60×d frames each representing 60 samples with d features. Frames are fed into four consecutive convolution layers with standard rectified ...
Nafees Ahmed's user avatar
1 vote
1 answer
23 views

LSTM with multiple data streams

I am working on the following problem: I have ~10 weather stations in somewhat approximate areas, at some points during the day (different for each station), I get readings of various data points (...
PenguinHook's user avatar
0 votes
1 answer
22 views

Sequential models and distribution shift in RL

We know the problem of "distribution shift" in deep Reinforcement Learning, where the change in policy during training affects the behavior of the agent and therefore the distribution of the ...
SuperTardigrade's user avatar
0 votes
0 answers
20 views

ROC curve for multiclassification - results sound not correct

I'm working on a multiclassification task using LSTM algorithm, i generated my roc curve plots but they give scores like 1 , 0.99, 0.97 however i have an accuracy of 0.97, Precision 0.65, Sensitivity/...
biihu's user avatar
  • 1
0 votes
0 answers
41 views

How to setup correctly a sequence generation task with RL/policy gradient learning?

I've a pretrained model for sequence generation that I'd like to improve with RL but there are several shady points. So, I have the following model and loss function: ...
eris's user avatar
  • 1
0 votes
0 answers
41 views

Updating custom output layers of an LSTM network

I have a text generation task learning to predict the next word with an LSTM network with multiple output layers. After the generation of a sentence has finished, I calculate a reward for the whole ...
eris's user avatar
  • 1
0 votes
1 answer
21 views

Seq2Seq model- Confusing about the dimension of Seq2Seq model [closed]

I am new to Seq2Seq and hope to find a proper guildances, advices. I am doing a Project from an online course so I can not give the material but I got my Project notebook on Github I want to ask ...
QH.Chu's user avatar
  • 3
0 votes
0 answers
18 views

How do I find a similar RNN as a starting point?

I am new to machine learning and neural networks and I want to create a neural network for a study project. I would like to create a RNN, that uses one (A) or several time series (with the same length,...
xlaub's user avatar
  • 1
0 votes
0 answers
47 views

What are the *non-cost-related* reasons RNN+Attention underperform Transformers?

There are obvious trainability and performance challenges with RNNs, such as having to process in serial and BPTT. But let's say we magically had an "optimal" set of weights for the RNN + ...
llllvvuu's user avatar
  • 146
1 vote
1 answer
40 views

How does the linear layer step work and what should I do at the end of the LSTM?

So basically I've read some text about LSTM, and luckily they mentioned the linear layer step at the end of the LSTM Process. However, they didn't explain how it works or what I would need to convert ...
Anish Kommireddy's user avatar
1 vote
0 answers
16 views

good period prediction but bad magnitude using keras LSTM neural network

I want to predict the voltage of a battery along time using neural networks. This voltage is read using an ADC and generates a charge/discharge profile that ideally looks like this: Which goes very ...
bardulia's user avatar
  • 111
0 votes
1 answer
45 views

In theory, LSTM should be adequate to solve my problem, but it doesn't

I am currently doing my Master's Thesis in the area of wireless sensing using radio waves. I have 4 receiver distributed in the corners of my room and each of them records so called Channel Impulse ...
binaryBigInt's user avatar
0 votes
0 answers
29 views

LSTM Ensemble: Combine low, mid, and high frequency time series data

I am trying to implement time series classification, but I am struggling a bit with the fact that my multivariate data has mixed frequencies. I have about 10 variables that are updated every minute, ...
Ai4l2s's user avatar
  • 121
0 votes
2 answers
257 views

LUT-Based Sigmoid and Tanh Activation-Functions in Integer Quantized Networks

I want to understand how activation functions, specifically tanh and sigmoid, are used in int8 quantized neural networks. Even more specific, I want to understand a Look-up-Table based approach. My ...
Necrotos's user avatar
0 votes
0 answers
11 views

Can I combined the trained model between different source but same model structure?

Here I got two different deep-learning models that were trained by LSTM and time-series data. The data is the usage percentage of CPU from two different computers. Each computer job was the same. It ...
orde.r's user avatar
  • 103
0 votes
0 answers
39 views

Not pre-trained binary transformer model

I stacked with a problem. My default Transformer model totally does not learn how to evaluate python logical expressions, like: '(False and not True) xor False or (not False and False)'. Model should ...
Oleksandr Ovcharenko's user avatar
0 votes
1 answer
583 views

Which preprocessing is the correct way to forecast time-series data using LSTM?

I just started to study time-series forecasting using RNN. I have a few months of time series data that was an hour unit. The data is a kind of percentage value of my little experiment and I would ...
orde.r's user avatar
  • 103
0 votes
0 answers
34 views

Is this lstm diagram correct?

I made an LSTM diagram, but do not know if it is correct. Can you point out any errors in case there are any?
David H. J.'s user avatar
0 votes
0 answers
17 views

Tips for getting LSTM to train for next word predcition

I am trying to train an LSTM network for next work prediction. I have scraped a rather large dataset from Wikipedia of country descriptions. I have done normal preprocessing (removing punctuation and ...
thehumaneraser's user avatar
1 vote
0 answers
41 views

The model's accuracy becomes suddenly so unreasonably good at beginning of the training process. I need an explaination

I am practicing machine translation using seq2seq model (more specifically with GRU/LSTM units). The following is my first model: This model first archived about 0.03 accuracy score and gradually ...
Đạt Trần's user avatar
0 votes
1 answer
24 views

Regression Model diverging after adding a new feature with higher variance and magnitude

In a time series regression problem I'm predicting "change" rather than the actual intended value i.e Instead of: ...
Darren Rahnemoon's user avatar
1 vote
1 answer
53 views

Transfer Learning for Solar Energy Production Forecasting with LSTM: Generalized vs. Specialized Models

I am working on a solar energy production forecasting problem using LSTM multi-step models to predict 1/4/8h ahead of solar energy production for different solar installations. Our goal is to help ...
Guilherme Vieira's user avatar
1 vote
1 answer
69 views

Recognize patterns within random sequences

I am familiar with ANNs as I studied them back in the days for regression and currently I'm working with CNN's for image recognition. But recently I was reading more about pattern recognition in ...
FELIPE_RIBAS's user avatar
0 votes
1 answer
566 views

Difference between sequence length and hidden size in LSTM

It does not come clear to me how the seq_length is not the exact same as the hidden_size in LSTMs. For example, in the next ...
moth123's user avatar
  • 31
3 votes
2 answers
1k views

Can LSTM model use ReLU or LeakyReLU as the activation funtion?

Can LSTM model use ReLU or LeakyReLU as the activation funtion? If so, when should one use tanh and when should one use ReLU or LeakyReLU?
BlueSnake's user avatar
1 vote
1 answer
121 views

What should I do, reinforcement learning agent gives different result on every train?

I'm using PPO+LSTM to create a trading bot. The agent is trained on 3 years of data and tested on 1 year. Every time I train the agent with same set of hyper-parameters, I get very different results ...
ad124j2's user avatar
  • 11
1 vote
1 answer
35 views

Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?

I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
Krusty the Clown's user avatar
0 votes
1 answer
90 views

Why final memory state equals to the last hidden state of entire hidden state sequence?

when return_sequences=True and return_state=True, a TensorFlow LSTM outputs the hidden states of the LSTM cell along with the ...
noone's user avatar
  • 123
1 vote
0 answers
19 views

SCINet: how does interactive learning work?

i'm having some trouble understanding how does the basic building block of a SCINet works. In the paper the author describes the SCI-block with the following figure: In which $\phi$, $\theta$, $\eta$ ...
Juan Hirschmann's user avatar
0 votes
0 answers
26 views

How can I transform the LSTM output to an embedding matrix of actions?

Page 3 of the paper Feudal Networks for Hierarchical Reinforcement Learning describes producing an 'embedding matrix' $U$ of size $(|a| \times k)$ from the output of an LSTM, given a $(d \times 1)$ ...
Telf's user avatar
  • 1
0 votes
0 answers
17 views

Text Classification Model unable to learn

I am trying to build a text classification model. When I train the model it is unable to improve accuracy and at some point accuracy even decreases and loss increases. I have researched for possible ...
javi11br's user avatar
0 votes
1 answer
63 views

How to identify overfitting in LSTM-RNN using metrics?

How can I identify overfitting on a RNN-LSTM with the following metrics: RMSE, MSE, RAE, R-squared ? I have searched papers and google results. I don't see something clear to my mind. Also I rarely ...
just_learning's user avatar
0 votes
0 answers
29 views

Modeling the previous inputs to affect next output in Machine learning

I am working on a dataset contains one output variable and a number of input variables.The data looks like the following: Y, X1, X2, X3, X4 7, 5, 0.7, 8, 9 3, 6, 0.3, 9, 9 .... Where Y is the output ...
Yazan Alatoom's user avatar
0 votes
0 answers
32 views

LSTM with time-series data transform

I have a time series data which has distinct time steps. For example, one time series data was recorded with the 1/30(min) time step, but some other data may have 1/15(min), 1/6(min), 1/5(min) time ...
user124697's user avatar
1 vote
0 answers
28 views

bad prediction when having noise on the data: LSTM time-series regression

I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
stack offer's user avatar
1 vote
0 answers
14 views

Where exactly is permutation happening in equation 5 of the paper "Learning with Sets in Multiple Instance Regression Applied to Remote Sensing"?

I am reading the article Learning with Sets in Multiple Instance Regression Applied to Remote Sensing about creating an embedding which is order-invariant to inputs ($m_{l}$). They referred to order-...
Oculu 's user avatar
  • 43
2 votes
2 answers
2k views

Val loss doesn’t decrease after a certain number of epochs

I’m working on a classification problem (500 classes). My NN has 3 fully connected layers, followed by an LSTM layer. I use nn.CrossEntropyLoss() as my loss ...
helloworld's user avatar
0 votes
1 answer
140 views

Extremely good results in RNN-LSTM python code!! How can this happen?

I am using this code here: https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/ and more specifically the python code under the (1st) paragraph "...
just_learning's user avatar
1 vote
2 answers
113 views

How does Seq2Seq with attention actually use the attention (i.e. the context vector)?

For neural machine translation, there's this model "Seq2Seq with attention", also known as the "Bahdanau architecture" (a good image can be found on this page), where instead of ...
Mew's user avatar
  • 181
0 votes
1 answer
99 views

Does sequence matter in LSTM?

In general, as long as the items are in order as per time sequence, does it matter if it's in ascending time sequence or descending time sequence for LSTM within the input vector? eg: ABCDEF -> G ...
Nibras Reeza's user avatar
0 votes
1 answer
112 views

Choice of LSTM for price prediction

I have a dataset with features (f) for different stocks (S) and want to infer for price using an LSTM model. Here is my df: year S1_price S1_f1 S1_f2 S2_price S2_f1 S2_f2 Sn_price Sn_f1 Sn_f2 2010 ...
Sphenoidale's user avatar
0 votes
1 answer
32 views

What is the training accuracy of this model?

I’m trying to classifiy ECG signals using LSTM and MATLAB, the above plot shows that the training accuracy of the system is 100% but when I apply this code to calculate and get the accuracy I get only ...
LinkToPhD's user avatar
0 votes
1 answer
78 views

Limitations of LSTMs

I'm training an LSTM model for classification on accelerometer data, and I get better results when I downsample the signal to 25 Hz than when I use a 50 Hz signal. I use the same time frame of 1.5 ...
Haffi112's user avatar
  • 121
0 votes
0 answers
93 views

How is it possible to use batches of data from within the same sequence with an LSTM?

ETA: More concise wording: Why do some implementations use batches of data taken from within the same sequence? Does this not make the cell state useless? Using the example of an LSTM, it has a hidden ...
Recessive's user avatar
  • 1,346
2 votes
2 answers
124 views

LSTM exploding? - multiple parallel time series with multiple variables

I have the following situation: Stock Time_Stamps Feature_1 Feature_2 Feature_n Price Stock_1 2019 0.5 1.0 1.0 100 Stock_1 2020 0.7 1.3 0.9 90 Stock_2 2019 0.3 0.9 1.1 110 Stock_2 2020 0.2 0.8 1....
Sphenoidale's user avatar

1
2 3 4 5 6