Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [lstm]

For questions about LSTM (long-short term memory) networks.

1
vote
1answer
15 views

Neurological interpretation of LSTMs

I am searching for an Interpretation of LSTMs and recurrent neural Networks within Cognitive Neuroscience. Is there a mechanism in the human brain, that works analog to LSTMs? How does Long-term ...
0
votes
0answers
5 views

LSTM output dimensionality

I am new to LSTMs. When reading the papers and websites about LSTM architecture, there is something I do not get. As I understand it, a single LSTM layer can have multiple LSTM cells (just like a ...
0
votes
0answers
12 views

Did anyone try topic modelling with neural nets?

I constantly see Latent Dirichlet Allocation (LDA) as a go to technique for topic modelling. It performs okay-ish, but ignores word context and (subjectively) seems outdated. Has anyone implemented ...
0
votes
1answer
19 views

Reduce the effect of excessive zeros

I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I ...
1
vote
1answer
40 views

Most efficient neural network for human activity recognition

A paper from machinelearningmastery.com on human activity recognition states that 1D convolutional neural networks work the best on classification of human activities using data from accelometer. But, ...
0
votes
0answers
7 views

Why does an RNN for text generation benefit from having more nodes than sequence length

If I have an RNN for text generation and want the RNN to learn characterwise, I partition the text to learn into sequences of equal length. Now my logic would be: If I have a sequence length x, then I ...
0
votes
0answers
14 views

Zero inflated binomial for excessive zeros

That photo shows that we have a trend. However, the trend is not exceptional because there is too much zeros in the target set and the features. MSE Loss / criterion on data $(x,y)$ Current ...
1
vote
0answers
13 views

Learning Tree Paths when Given Vectors

The problem statement: Mapping from a vector space representation onto a tree structure. Possible solution: Given a word vector as input, produce a path in the tree from the root down to the node ...
1
vote
0answers
23 views

Price difference predictions curve almost vanished

With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well ...
1
vote
1answer
94 views

Price Movement Forecasting Issue

I am working on a project for price movement forecasting and I am stuck with poor quality predictions. At every time-step I am using an LSTM to predict the next 10 time-steps. The input is the ...
1
vote
0answers
17 views

How to wire up bi-directional LSTMs?

I am interested in creating the neural network architecture described in this recent paper. However, I am not sure how to get started. Does TFLearn or Keras allow us to wire the LSTM in such ways?
2
votes
0answers
18 views

If there are several computers on a subnet, can training time be reduced by distributing the work across them?

We have multiple computers and the ability to ssh between them. What are options using either Java, C/C++, JavaScript, or Python to distribute our learning tasks? We will be using DCNN, DQN, and LSTM ...
1
vote
0answers
18 views

How to understand marginal loglikelihood objective function as loss function (explanation of an article)?

I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned ...
2
votes
1answer
66 views

Difficulty understanding Keras LSTM fitting data

I'm try to train a RNN with a chunk of audio data, where X and Y are two audio channels loaded into numpy arrays. The objective is to experiment with different NN designs to train them to transform ...
0
votes
0answers
18 views

Sequentia data with static data (gender, group, Major) in RNN model (LSTM)

I have a sequential dataset and I built an LSTM model using Keras for prediction accuracy. I would like to add static data ( i.e. gender, blood group, major) to the sequential data. I found a paper (...
1
vote
0answers
34 views

Structure of a multilayered LSTM neural network?

I implemented a LSTM neural network in Pytorch. It worked but I want to know if it worked the way I guessed how it worked. Say there's a 2-layer LSTM network with 10 units in each layer. The inputs ...
3
votes
1answer
49 views

Where can I find the original paper that introduced RNNs?

I was able to find the original paper on LSTM, I was not able to find the paper that introduced "vanilla" RNNs. Where can I find it?
2
votes
1answer
60 views

Why would giving my AI more data make it perform worse?

So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample. Next I tried to get it to generate limericks using these 100,000 limericks. It generated ...
0
votes
0answers
54 views

Deep learning with a lot of training data

I am building a bidirectional LSTM to do a sequential text-tagging task (particularly, automatic punctuation). Usually, the training is done in iterations, where in each iteration, the entire training ...
0
votes
1answer
74 views

LSTM RNN structure

I am currently looking into LSTMs. I found this nice blog post, which is already very helpful, but still, there are things I don't understand, mostly because of the collapsed layers. The input $X_t$,...
3
votes
1answer
304 views

How does LSTM in deep reinforcement learning differ from experience replay?

In the paper Deep Recurrent Q-Learning for Partially Observable MDPs, the author processed the Atari game frames with an LSTM layer at the end. My questions are: How does this method differ from the ...
0
votes
0answers
45 views

Speech recognition/ Voice Recognition: How to use NLP after a speech-to-text to improve accuracy?

I am planning to use Mozilla DeepSpeech for the project. For the use case at hand, I'm thinking if there's a possibility of using any NLP engine after the speech-to-text to improve its efficiency. If ...
3
votes
1answer
42 views

Using different timesteps for features and target value

I would like to know whether it's wrong; when working with time series data; to use daily prices as features and the price after 3 days as target. Is this correct or should I use the next-day price ...
4
votes
1answer
61 views

What would be the best approach to teach an AI to learn how to “sing” along a beat?

I have heard and read about HyperGAN, LSTM and a few other techniques, but I have a hard time piecing the overall concept together. End Goal Being able to input an instrumental and get an output of ...
1
vote
0answers
45 views

Using CNN LSTMs for prediction of images from image series

I have the following setup for a prediction task: I want to predict entire pictures from previously given pictures. In my case, only 2 pixels in every frame are neither black nor white, they are some ...
1
vote
0answers
50 views

Generating time series for doing time-series forecasting with LSTM

I have a .db file with columns as described below. This data has been collected by a software which monitors the file usage in a filesystem or in other words generates metadata about all the files in ...
1
vote
0answers
21 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
1
vote
1answer
133 views

Backward Pass for LSTMs

TL;DR I am currently trying to understand the mathematics in Ger's paper Long Short-Term Memory in Recurrent Neural Networks. I have found the document clear and readable so far. On pg. 21 of the ...
4
votes
2answers
1k views

Can LSTM Nets be speed up by GPU?

I am training LSTM Nets with Keras on a small mobile GPU. The speed on GPU is slower then on CPU. I found some articles that say that it is hard to train LSTMs (RNNs) on GPUs because the training ...
1
vote
0answers
19 views

How to visualize/interpret text prediction model results?

I am using LSTM model to predict the next xml markup from an input seed. I have trained my model on 1500 xml files. Each xml file is generated randomly. I am wondering if there is a way to visualize ...
3
votes
1answer
206 views

Structure of LSTM RNNs

I have some very basic questions here. This is probably because I didn't read the relevant documents closely enough. If I used some terminology incorrectly, please point them out. Thank you! For ...
1
vote
0answers
84 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
3
votes
2answers
45 views

How should my output layer activate for a sequence to sequence LSTM model with word2vec output

I'm having trouble grasping how to output word vectors from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot ...
0
votes
0answers
172 views

Load forecasting using LSTM in Matlab 2018a

I am using MATLAB 2018a, and looking for an explanation, or pointers, on how can I modify the matlab example for "sequence to sequence regression" to use it with "double" type predictor data array ...
3
votes
1answer
152 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
1
vote
0answers
58 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
5
votes
4answers
180 views

Use Machine/Deep Learning to Guess a String

I want to be able to input a block of text and then have it guess a string within a predefined range (i.e. a string that starts with three letters and ends with five numbers like "XXX12345", etc). ...
1
vote
2answers
62 views

Time Series: LSTM or Augmented Vector Space?

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors. There's two that I'm considering. An LSTM or ...
1
vote
0answers
36 views

Learning from events. Supervised, Unsupervised or MDP?

I have a large set of simulation logs for a market simulation of which I want to learn from. The market includes: customers products (subscriptions) The customers choose products and then stick with ...
3
votes
1answer
292 views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
0
votes
0answers
562 views

R-CNN LSTM (using Detectron) for image caption task

I am new to deep learning. I have run the caffe2 tutorials to train the LeNet for Mnist with my dataset and also the LSTM tutorial. Initially I was thinking of combining both systems to get a CNN ...
2
votes
1answer
551 views

What is the difference between ConvLSTM and CNN LSTM?

What will be the difference when used for video classification? Will they yield different results or are they the same fundamentally?
2
votes
1answer
107 views

How are LSTM's trained for text generation?

I've seen some articles about text generation using LSTMs (or GRUs) for text generation. Basically it seems you train them by folding them out, and putting a letter in each input. But say you trained ...
2
votes
0answers
64 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
7
votes
5answers
6k views

Using Machine/Deep learning for guessing Pseudo Random generator

Is it possible to feed a neural network, the output from a random number generator and expect it learn the hashing/generator function. So that it can predict what will be the next generated number? ...
6
votes
2answers
131 views

Shortening the development time of a neural network

I am developing an LSTM for sequence tagging. During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. After ...
2
votes
0answers
47 views

seq2seq vector to letters model

I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
2
votes
1answer
164 views

The state of a recurrent neural network

I am Reading "Supervised Sequence Labelling with Recurrent Neural Networks" written by Alex Graves to try to understand LSTM networks and I am a bit confused about the equations. Specifically, what I ...
1
vote
1answer
63 views

Multi-param LSTM input

Lets say you install your LSTM machine on a road between London and Oxford. And it makes observations. A car with 3 people inside drives past it in one direction 21 sec after previously observed car (...
11
votes
4answers
18k views

How to select number of hidden layers and number of memory cells in LSTM

As I am learning about LSTMs, but also about neural networks in general, I am trying to find some existing research on how to select the number of hidden layers and the size of these. Is there an ...