Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

This tag should be used for posts dealing with LSTM networks.

2
votes
1answer
31 views

Using different timesteps for features and target value

I would like to know whether it's wrong; when working with time series data; to use daily prices as features and the price after 3 days as target. Is this correct or should I use the next-day price as ...
4
votes
1answer
56 views

What would be the best approach to teach an AI to learn how to “sing” along a beat?

I have heard and read about HyperGAN, LSTM and a few other techniques, but I have a hard time piecing the overall concept together. End Goal Being able to input an instrumental and get an output of ...
1
vote
0answers
37 views

Using CNN LSTMs for prediction of images from image series

I have the following setup for a prediction task: I want to predict entire pictures from previously given pictures. In my case, only 2 pixels in every frame are neither black nor white, they are some ...
1
vote
0answers
17 views

Generating time series for doing time-series forecasting with LSTM

I have a .db file with columns as described below. This data has been collected by a software which monitors the file usage in a filesystem or in other words generates metadata about all the files in ...
1
vote
0answers
15 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
1
vote
1answer
57 views

Backward Pass for LSTMs

TL;DR I am currently trying to understand the mathematics in Ger's paper Long Short-Term Memory in Recurrent Neural Networks. I have found the document clear and readable so far. On pg. 21 of the ...
3
votes
2answers
89 views

Can LSTM Nets be speed up by GPU?

I am training LSTM Nets with Keras on a small mobile GPU. The speed on GPU is slower then on CPU. I found some articles that say that it is hard to train LSTMs (RNNs) on GPUs because the training ...
1
vote
0answers
17 views

How to visualize/interpret text prediction model results?

I am using LSTM model to predict the next xml markup from an input seed. I have trained my model on 1500 xml files. Each xml file is generated randomly. I am wondering if there is a way to visualize ...
3
votes
1answer
47 views

Structure of LSTM RNNs

I have some very basic questions here. This is probably because I didn't read the relevant documents closely enough. If I used some terminology incorrectly, please point them out. Thank you! For ...
1
vote
0answers
31 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
2
votes
2answers
32 views

How should my output layer activate for a sequence to sequence LSTM model with word2vec output

I'm having trouble grasping how to output word vectors from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot ...
0
votes
0answers
76 views

Load forecasting using LSTM in Matlab 2018a

I am using MATLAB 2018a, and looking for an explanation, or pointers, on how can I modify the matlab example for "sequence to sequence regression" to use it with "double" type predictor data array ...
3
votes
1answer
78 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
1
vote
0answers
36 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
3
votes
3answers
97 views

Use Machine/Deep Learning to Guess a String

I want to be able to input a block of text and then have it guess a string within a predefined range (i.e. a string that starts with three letters and ends with five numbers like "XXX12345", etc). ...
1
vote
2answers
48 views

Time Series: LSTM or Augmented Vector Space?

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors. There's two that I'm considering. An LSTM or ...
1
vote
0answers
27 views

Learning from events. Supervised, Unsupervised or MDP?

I have a large set of simulation logs for a market simulation of which I want to learn from. The market includes: customers products (subscriptions) The customers choose products and then stick with ...
3
votes
1answer
164 views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
0
votes
0answers
319 views

R-CNN LSTM (using Detectron) for image caption task

I am new to deep learning. I have run the caffe2 tutorials to train the LeNet for Mnist with my dataset and also the LSTM tutorial. Initially I was thinking of combining both systems to get a CNN ...
3
votes
1answer
303 views

What is the difference between ConvLSTM and CNN LSTM?

What will be the difference when used for video classification? Will they yield different results or are they the same fundamentally?
2
votes
1answer
85 views

How are LSTM's trained for text generation?

I've seen some articles about text generation using LSTMs (or GRUs) for text generation. Basically it seems you train them by folding them out, and putting a letter in each input. But say you trained ...
2
votes
0answers
45 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
4
votes
3answers
4k views

Using Machine/Deep learning for guessing Pseudo Random generator

Is it possible to feed a neural network, the output from a random number generator and expect it learn the hashing/generator function. So that it can predict what will be the next generated number? ...
5
votes
2answers
122 views

Shortening the development time of a neural network

I am developing an LSTM for sequence tagging. During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. After ...
2
votes
0answers
39 views

seq2seq vector to letters model

I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
2
votes
1answer
160 views

The state of a recurrent neural network

I am Reading "Supervised Sequence Labelling with Recurrent Neural Networks" written by Alex Graves to try to understand LSTM networks and I am a bit confused about the equations. Specifically, what I ...
1
vote
1answer
47 views

Multi-param LSTM input

Lets say you install your LSTM machine on a road between London and Oxford. And it makes observations. A car with 3 people inside drives past it in one direction 21 sec after previously observed car (...
8
votes
4answers
12k views

How to select number of hidden layers and number of memory cells in LSTM

As I am learning about LSTMs, but also about neural networks in general, I am trying to find some existing research on how to select the number of hidden layers and the size of these. Is there an ...
2
votes
0answers
392 views

Text Categorization using LSTM, word embeddings

I am currently reading the paper "Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings", and I have some difficulties understanding some of their simplifications to an ...
7
votes
1answer
3k views

How to train a chatbot

I wanted to started experimenting with neural network and as a toy problem I wished to train one to chat, i.e. implement a chatting bot like cleverbot. Not that clever anyway. I looked around for ...
1
vote
2answers
349 views

In LSTM text generation can low amount of training data be compensated?

I'm playing with an LSTM to generate text. In particular, this one: https://raw.githubusercontent.com/fchollet/keras/master/examples/lstm_text_generation.py It works on quite a big demo text set ...