Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

Filter by
Sorted by
Tagged with
1
vote
0answers
13 views

Do RNN solves the need for LSTM and/or multiple states in Deep Q-Learning?

Introduction I am trying to setup a Deep Q-Learning agent. I have looked that the papers Playing Atari with Deep Reinforcement Learning as well as Deep Recurrent Q-Learning for Partially Observable ...
1
vote
0answers
17 views

length independent sequence classification methods

I am looking to do sequence classification using deep learning. The length of my sequences can vary from a few hundred to several tens of thousands of characters. I was wondering what is a good ...
1
vote
0answers
9 views

Building a template based NLG system to generate a report from data

I am a newbie to NLP and NLG. I am tasked to develop a system to generate a report based on a given data table. The structure of the report and the flow is predefined. I have researched on several ...
3
votes
0answers
17 views

Is there any way of generating fixed-length sequences with RNNs?

Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my ...
2
votes
1answer
44 views

What are pros and cons of Bi-LSTM as compared to LSTM?

What are the pros and cons of LSTM vs Bi-LSTM in language modelling? What was the need to introduce Bi-LSTM?
2
votes
0answers
27 views

How to predict an event (or action) based on a window of time-series measurements?

I have an input vector $X$, which contains a series of measurements within a period, e.g. 100 measurements in 1 sec. The goal is to predict an event, let's say, moving forward, backward or static. I ...
2
votes
3answers
105 views

Using sigmoid in LSTM network for multi-step forecasting

I'm trying to develop a multistep forecasting model using LSTM Network. The model takes three times steps as input and predicting two time_steps. both input and output columns are normalised using ...
1
vote
1answer
41 views

How does the number of stacked LSTM layers or units in each layer affect the model complexity?

I playing around sequence modeling to forecast the weather using LSTM. How does the number of layers or units in each layer exactly affect the model complexity (in an LSTM)? For example, if I ...
1
vote
0answers
20 views

Model for supervised sequence classification task

The Problem I am currently working on a sequence classification problem I try to solve with machine learning. The target variable is the current state of a system. This target variable is following a ...
2
votes
0answers
21 views

Can the cross-entropy loss be used for a NLP task with LSTM?

I am trying to build an LSTM model to generate Shakspeare-like poems. I have training set $\{s_1,s_2, \dots,s_m\}$, which are sentences of Shakespeare poems, and each sentence contains words $\{w_1,...
1
vote
1answer
45 views

Are sentences from the same document independent and identically distributed?

I am trying to build an LSTM model to generate Shakspeare-like poems. I have data set $\{s_1, s_2, \dots, s_m \}$, which are sentences of Shakespeare poems, and each sentence contains words $\{w_1, ...
0
votes
0answers
9 views

How should I go about selecting an optimal num_units within a LSTM cell for different sequence sizes

I am currently working on a stock market prediction model which incorporates sentiments along with historical price for next day price prediction. I wanted to test different window / sequence size e....
0
votes
0answers
9 views

My CTC loss model's loss stagnates and then outputs only blank characters

I am trying to implement BaiDu's DeepSpeech1 in keras using CTC loss, my code is below: ...
1
vote
0answers
11 views

How to pad sequences during training for an encoder decoder model

I've got an encoder-decoder model for character level English language spelling correction, it is pretty basic stuff with a two LSTM encoder and another LSTM decoder. However, up until now, I have ...
1
vote
0answers
14 views

Recurrent neural Network for survival analyses: Dealing with forecast data as feature which can exceed the number of days untill a event occurs

I am building a Recurrent Neural network (LSTM) for predicting the number of days until a Pollen season starts (when the cumulative of the year exceeds X). One of the features I am including in my ...
0
votes
0answers
13 views

Feed data into Keras LSTM layer

I'm trying to understand how to feed data into LSTM layer of Keras, but I'm in trouble and I don't understand how to do it. I've ...
1
vote
1answer
170 views

What’s the difference between LSTM and RNN?

What's the difference between LSTM and RNN? I know that RNN is a network layer used in neural networks, but what exactly is an LSTM? Is it also a network layer with the same characteristics?
1
vote
0answers
17 views

Can tensorflow debugger debug a trained keras model during prediction?

I have a trained LSTM model created with keras. Is it possible to use tensorboard debugger or tensorflow debugger to debug the model during prediction runtime? Meaning that it steps through the model ...
1
vote
0answers
15 views

When stacking LSTM's, should the hidden units increase?

I'm using Weights and Biases to do some hyperparameter sweeping for a supervised sequence-to-sequence problem I'm working on. One thing I noticed is that the sweeps with a gradually increasing number ...
1
vote
0answers
13 views

Simple sequential model with LSTM which doesn't converge

I'm actually trying to create a sequential neural network in order to translate a "human" sentence in a "machine" sentence understandable by an algorithm. Like It didn't work, I've try to create a NN ...
0
votes
1answer
27 views

time-series prediction : loss going down, then stagnates with very high variance

I am trying to design a model based on LSTM cells to do time-series prediction. The ouput value is an integer in [0,13]. I have noticed that one-hot encoding it and ...
1
vote
0answers
29 views

Are there any tools I can use to debug a Keras LSTM model during prediction?

I want to be able to debug my Keras LSTM model. For example, I want to be able to check the values of the input/output gates, cell states and hidden states at every time-step during prediction. Are ...
0
votes
0answers
10 views

What will be the sequence of steps in a human activity recognition model using LSTM?

In the context of these steps detection, tracking, action classification and activity recognition. Which step will be first and further sequence?
0
votes
0answers
14 views

RNN/LSTM with a large amounts of data

I have sequence data that's quite large - 4x65k per sample. I'm interested in doing classification problems. The number of classes is moderate - ~27 or so What is the suggestion for dealing with this ...
2
votes
0answers
19 views

How do I make my LSTM model more sensitive to changes in the sequence?

I have a many to one LSTM model for multiclass classification. For reference, this is the architecture of the model ...
1
vote
1answer
18 views

LSTM model on different time scales

I am a newbie to machine learning. I have an LSTM model that predicts the next output n+1 time 1, params 1, output 1 time 2, params 2, output 2 time 3, params 3, output 3 . . time n, params n, , ...
0
votes
0answers
12 views

Which hidden state should I use for a trajectory when incorporating LSTM into RL?

I'm trying to wrap my head around using LSTM in an RL algorithm like actor-critic or PPO. I've found this Github code which presents this in a very simple manner, however I have a very limited ...
0
votes
1answer
34 views

LSTM implementation in KERAS

I would like to build an LSTM to predict the correct words order given a sentence. My dataset is composed of sentences, where each sentence has a variable number of words (each word is embedded). The ...
3
votes
1answer
63 views

Is the LSTM component a neuron or a layer?

Given the standard illustrative feed-forward neural net model, with the dots as neurons and the lines as neuron-to-neuron connection, what part is the (unfold) LSTM cell (see picture)? Is it a neuron (...
0
votes
0answers
21 views

Web stream requests prediction architecture

What's in your opinion the best possible architecture for the following problem ? If you have any code that can be used it would be great . Dataset : 400.000 samples given in hex format in an .xlsx ...
2
votes
1answer
47 views

LSTM convergence

Despite the problem being very simple, I was wondering why a LSTM network was not able to converge to a decent solution. ...
1
vote
0answers
17 views

Did people analyze dynamics of very simple LSTMs?

I wonder if researchers tried to understand how LSTMs work by analyzing dynamics of simple LSTM (e.g. with 2 units)? For example how hidden state evolves depending on the properties of weight matrices....
1
vote
1answer
59 views

How should I design this LSTM network to perform stock prediction?

I'm trying to develop a stock predictor. I'm using LSTM but I am unsure about the structure of the Neural Network. For example, I'm assuming that the Neural Network is a many-to-one since we have ...
0
votes
1answer
43 views

How to process data in a data stream for a LSTM

How can a data stream for a RNN (LSTM) be handled, when the stream contains data sets belonging to different prediction classes? Training phase: I have trained a LSTM to predict a class out of a ...
4
votes
0answers
55 views

RNN models displays upper limit on predictions

I have trained a RNN, GRU, and LSTM on the same dataset, and looking at their respective predictions I have observed, that they all display an upper limit on the value they can predict. I have ...
3
votes
0answers
44 views

Why don't the neural networks inside LSTM cells contain hidden layers?

I watched a video explaining how LSTM cells have very rudimentary feed-forward neural networks, basically a 2 layer input-output with no hidden layers. Why don't LSTM cells have more complex neural ...
4
votes
1answer
43 views

Training an RNN to answer simple quesitons

I would like to train an RNN to follow the sentences: "Would you like some cheese"? with "Yes, I would like some cheese." So whenever the template "Would you like some ____?" appears then RNN ...
3
votes
0answers
31 views

How can I do hyperparameter optimization for a CNN-LSTM neural network?

I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not ...
3
votes
1answer
56 views

Can dropout layers not influence LSTM training?

I am working on a project that requires time-series prediction (regression) and I use LSTM network with first 1D conv layer in Keras/TF-gpu as follows: ...
3
votes
2answers
69 views

How to use LSTM to generate a paragraph

A LSTM model can be trained to generate text sequences by feeding the first word. After feeding the first word, the model will generate a sequence of words (a sentence). Feed the first word to get the ...
0
votes
0answers
26 views

Outliers detection problem in neural networks

Assuming we have big m x n input dataset with m x 1 output vector. It's a classification problem with only two possible values: either 1 or 0. Now the problem is that almost all elements of the output ...
8
votes
1answer
137 views

How does the forget layer of an LSTM work?

Can someone explain the mathematical intuition behind the forget layer of an LSTM? So as far as I understand it, the cell state is essentially long term memory embedding (correct me if I'm wrong), ...
3
votes
2answers
72 views

Why can't LSTMs keep track of the “important parts” of a sequence?

I keep reading about how LSTMs can't remember the "important parts" of a sequence which is why attention-based mechanisms are required. I was trying to use LSTMs to find people's name format. For ...
2
votes
0answers
22 views

How are batch statistics computed in Recurrent Batch Normalization?

I'm implementing recurrent BN per this paper in Keras, but looking at it and those citing it, a detail remains unclear to me: how are batch statistics computed? Authors omit explicit clarification, ...
1
vote
0answers
33 views

What is the correct input shape for my LSTM network?

My professor gave us a workshop where we have to do classification of a dataset of ECG signals between healthy and unhealthy types using LSTM. Each signal consists of 1,285 time steps. What my prof ...
3
votes
1answer
32 views

Can non-sequential deep learning models outperform sequential models in time series forecasting?

Can a CNN (or other non-sequential deep learning models) outperform LSTM (or other sequential models) in time series data? I know this question is not very specific, but I experienced this when ...
3
votes
0answers
81 views

Why do regression LSTMs learn high to low inputs significantly better than low to high?

The specific problem I have is learning the relation $x^2$. I have an array of 0 through 19 (input values) and a target array of 0, 1, 4, 9, 16, 25, 36 and so on all the way up to $19^2$=361. I have ...
1
vote
1answer
82 views

What is the difference between Kaldi and DeepSpeech speech recognition systems in their approach?

I would like to know how do Kaldi and DeepSpeech speech recognition systems differ algorithmically? Which one would be more accurate for continuous speech in time?
2
votes
1answer
19 views

How to represent integer values in sequence to sequence prediction task in encoder-decoder LSTM?

I have a large 2D grid having 30k rows and 35k columns, so a total of 30x35k grid cells. Each grid cell is represented by a unique integer number (identity of grid cell). I have several trajectories ...
1
vote
0answers
46 views

How to train LSTM score prediction with very little data? (Bounty to be added)

I am trying to make a text score prediction network, and my dataset have 500 samples only. I know there is a public dataset called the ASAP Dataset. I have tested my model ...