Questions tagged [long-short-term-memory]

For questions related to the long-short term memory (LSTM), which refers to a recurrent neural network architecture that uses LSTM units. The first LSTM unit was proposed in 1997 by Sepp Hochreiter and Jürgen Schmidhuber in the paper "Long-Short Term Memory".

58 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
54 views

dimensions of hidden layer and cell state layer in LSTM

I was following some examples to get familiar with tensorflow LSTM related api, but noticed that all LSTM initialization functions require only num_units parameter which denotes number of hidden units ...
3
votes
0answers
18 views

What evaluation metric are used for sequence-to-sequence prediction problems?

I am solving many sequence-to-sequence prediction problems using RNN/LSTM. What type of evaluation metrics can be used for sequence prediction problems? One metric is the mean squared error (MSE) ...
3
votes
2answers
28 views

Using a neural network to identify a stable region within a set of data?

I am working on a problem in which I am attempting to find a stable region in a spiral galaxy. The PI I'm working with asked me to use machine learning as a tool to solve the problem. I have created ...
3
votes
0answers
109 views

What can be considered a deep recurrent neural network?

In the paper Deep Recurrent Q-Learning for Partially Observable MDPs, the DRQN is described as DQN with the first post-convolutional fully-connected layer replaced by a recurrent LSTM. I have DQN ...
3
votes
0answers
34 views

How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?

It seems that stacking LSTM layers can be beneficial for some problem settings in order to learn higher levels of abstraction of temporal relationships in the data. There is already some discussion on ...
3
votes
0answers
123 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
2
votes
0answers
26 views

How does a Bidirectional RNN work?

Could it be possible to reach a similar output via feeding a unidirectional network with the original data and the data played backwards?
2
votes
1answer
44 views

What sort of Neural Network is best suited to predicting a future purchase?

I have previously implemented a Neural Network with Back-Propagation that was able to learn Tic-tac-toe and could go pretty well at Connect-4. Now I'm trying to do a NN that can make a prediction. ...
2
votes
0answers
18 views

How can I detect fast and slow motion in videos?

I'm trying to detect if a given video shot is fast or slow motion. Basically, I need to calculate a "video motion" score in a given video sequence, meaning how fast or slow motion the video is. For ...
2
votes
0answers
42 views

Spike detection in time series using Artificial Neural Networks

I'm quite new in ANNs. I intend to use ANNs for predicting spike points in time series right before they happen. I've already used LSTM for another scenario, and I know that they can be used in ...
2
votes
1answer
74 views

LSTM Architecture

There is plenty of literature describing LSTMs in a lot of detail and how to use them for multi-variate or uni-variate forecasting problems. What I couldn't find though, is any papers or discussions ...
2
votes
0answers
39 views

Why experience reply memory in DQN instead of a RNN memory?

I was trying to implement a DQN without experience reply memory, and the agent is not learning anything at all. I know from readings that experience reply is used for stabilizing gradients. But how ...
2
votes
0answers
40 views

RNN: Different test results on balanced and unbalanced data

I trained a recurrent neural network (if it matters - it contains three CuDNNLSTM cells and 3 Dense layers, Dropout = 0.2). The result of data preparation is one array of ~330.000 sequences. Each ...
2
votes
0answers
18 views

Multi-field text input for LSTM

I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using ...
2
votes
0answers
33 views

How does a neural network output text box location data?

I'm interested in creating a convolutional neural network or LSTM to locate text in an image. I don't want to OCR the text yet, just find the text regions. Yes, I know Tesseract and other systems can ...
2
votes
0answers
97 views

Structure of a multilayered LSTM neural network?

I implemented a LSTM neural network in Pytorch. It worked but I want to know if it worked the way I guessed how it worked. Say there's a 2-layer LSTM network with 10 units in each layer. The inputs ...
2
votes
0answers
94 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
2
votes
0answers
65 views

seq2seq vector to letters model

I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
1
vote
0answers
17 views

Why do regression LSTMs learn high to low inputs significantly better than low to high?

The specific problem I have is learning the relation $x^2$. I have an array of 0 through 19 (input values) and a target array of 0, 1, 4, 9, 16, 25, 36 and so on all the way up to $19^2$=361. I have ...
1
vote
1answer
13 views

How to represent integer values in sequence to sequence prediction task in encoder-decoder LSTM?

I have a large 2D grid having 30k rows and 35k columns, so a total of 30x35k grid cells. Each grid cell is represented by a unique integer number (identity of grid cell). I have several trajectories ...
1
vote
0answers
43 views

How to train LSTM score prediction with very little data? (Bounty to be added)

I am trying to make a text score prediction network, and my dataset have 500 samples only. I know there is a public dataset called the ASAP Dataset. I have tested my model ...
1
vote
0answers
26 views

Initial LSTM hidden state and cell

If we use LSTMCell from torch: The initial hidden and cell layers should be CONSTANT (from the first time you run the program) and saved right? Like random seeds? ...
1
vote
0answers
105 views

How to use the LSTM layer in PPO architecture?

What is the best way of using the LSTM layer in PPO architecture? Should I use them in the first layer of both actor and critic, or use them just before the final layer of these networks? Should I ...
1
vote
0answers
36 views

Keras CRNN implementation with multiple input images

Hello I am trying to implement a CRNN with multiple input images (in my context it is 6 images) This is a regression problem and output is two real value. And for the CNN block I am thinking of using ...
1
vote
1answer
81 views

Do I need LSTM units everywhere in the network?

I have recently begun researching LSTM networks, as I have finished my GA and am looking to progress to something more difficult. I believe I am using the classic LSTM (if that makes any sense) and ...
1
vote
0answers
14 views

Training LSTM with class imbalance

I need to train an LSTM on some light curves, in order to find a signal (there are 2 classes signal and background). However the signal (data points corresponding to signal) is around 100 times less ...
1
vote
0answers
29 views

Why validation performance is unstable for my LSTM based model (labelling problems)?

I have trained a recurrent neural network based on 1 stack of LSTM cells. I use it to solve a classification problem. The RNN cell has 48 hidden states. The output of the last unfolded LSTM cell is ...
1
vote
0answers
39 views

Predicting sine using LSTM: Small output range and delayed output?

I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict $0.5*sin(t + N)$ given $0.5*sin(t)$ as an exercise. I have tweaked the model, changing the output layer ...
1
vote
0answers
18 views

How to train chat bot on infinite non-stationary data?

I have continual simulated data of million sentences of two simulated persons talking to each other in a room and I want to model one of the persons speech. Now, during this period things in the room ...
1
vote
0answers
281 views

Experiment shows that LSTM does worse than Random Forest… Why?

LSTM is supposed to be the right tool to capture path-dependency in time-series data. I decided to run a simple experiment (simulation) to assess the extent to which LSTM is better able to understand ...
1
vote
0answers
25 views

Kera's (normal) LSTM uses the GPU?

I'm running Kera's LSTM (not CuDNNLSTM) but I notice my GPU is under load. I need recurrent dropout, so I can only stick with <...
1
vote
0answers
35 views

how to estimate which item is bought most using AI

Problem: Maintaining the products is a big task for a retailer. If we can estimate using AI to predict which products will sell the most, we can maintain sufficient stocks of the product without ...
1
vote
0answers
221 views

Number of nodes in hidden layer for LSTM

I am starting to learn LSTM by understanding how it is used for creating a char-RNN and had a fundamental question. Does the number of nodes in the hidden layer need to be the same as that of the ...
1
vote
0answers
38 views

How could I learn tree paths given word embeddings?

I need to map from a vector space representation onto a tree structure. A possible solution: given a word vector as input, produce a path in the tree from the root down to the node that most closely ...
1
vote
1answer
71 views

Price difference predictions curve almost vanished

With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well ...
1
vote
0answers
22 views

How to wire up bi-directional LSTMs?

I am interested in creating the neural network architecture described in this recent paper. However, I am not sure how to get started. Does TFLearn or Keras allow us to wire the LSTM in such ways?
1
vote
0answers
21 views

If there are several computers on a subnet, can training time be reduced by distributing the work across them?

We have multiple computers and the ability to ssh between them. What are options using either Java, C/C++, JavaScript, or Python to distribute our learning tasks? We will be using DCNN, DQN, and LSTM ...
1
vote
0answers
41 views

How to understand marginal loglikelihood objective function as loss function (explanation of an article)?

I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned ...
1
vote
1answer
103 views

Difficulty understanding Keras LSTM fitting data

I'm try to train a RNN with a chunk of audio data, where X and Y are two audio channels loaded into numpy arrays. The objective is to experiment with different NN designs to train them to transform ...
1
vote
0answers
29 views

Has anyone investigated iteration awareness beyond RNN and LSTM?

This question considers the convergence of an artificial networks (MLPs, RNNs, LSTM nets, CNNs) over time or over the course of epochs made up of iterations through training examples. In this ...
1
vote
0answers
71 views

Using CNN LSTMs for prediction of images from image series

I have the following setup for a prediction task: I want to predict entire pictures from previously given pictures. In my case, only 2 pixels in every frame are neither black nor white, they are some ...
1
vote
0answers
158 views

Generating time series for doing time-series forecasting with LSTM

I have a .db file with columns as described below. This data has been collected by a software which monitors the file usage in a filesystem or in other words generates metadata about all the files in ...
1
vote
0answers
29 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
1
vote
0answers
29 views

How to visualize/interpret text prediction model results?

I am using LSTM model to predict the next xml markup from an input seed. I have trained my model on 1500 xml files. Each xml file is generated randomly. I am wondering if there is a way to visualize ...
1
vote
0answers
134 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
1
vote
0answers
56 views

Learning from events. Supervised, Unsupervised or MDP?

I have a large set of simulation logs for a market simulation of which I want to learn from. The market includes: customers products (subscriptions) The customers choose products and then stick with ...
1
vote
2answers
66 views

Why can't LSTMs tell a long story?

There is a recent trend in people using LSTMs to write novels. I haven’t attempted this myself. From what I’m hearing, they can tell a story, but it seems they lose the context of the story rather ...
0
votes
0answers
19 views

Why is the value range of my LSTM model's prediction different from my test labels?

I am using LSTM to do time-series anomaly detection. The data is an hourly sensor input across multiple years (i.e. the global_active_energy attribute of the dataset from https://www.kaggle.com/uciml/...
0
votes
0answers
14 views

Make an LSTM model for each class separately

I have a dataset of some activities. The dataset contains the status of different sensors and the label of activity. T trained a model in Keras with the following architecture which models the ...
0
votes
1answer
37 views

Preparation of input data

Tell me why my val_acc is always the same and how to solve this problem? I saw several topics on the Internet specifically on this problem but they did not help me (for example, use SGD with different ...