Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [lstm]

For questions about LSTM (long-short term memory) networks.

1
vote
0answers
11 views

If there are several computers on a subnet, can training time be reduced by distributing the work across them?

We have multiple computers and the ability to ssh between them. What are options using either Java, C/C++, JavaScript, or Python to distribute our learning tasks? We will be using DCNN, DQN, and LSTM ...
-1
votes
0answers
10 views

Classification and value prediction

Now I'm using deep learning for Human activity classification. Training data contains information like the following. acceleration, orientation, class and degree. Class is one of four activity: ...
1
vote
0answers
14 views

How to understand marginal loglikelihood objective function as loss function (explanation of an article)?

I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned ...
2
votes
0answers
30 views

Difficulty understanding Keras LSTM fitting data

I'm try to train a RNN with a chunk of audio data, where X and Y are two audio channels loaded into numpy arrays. The objective is to experiment with different NN designs to train them to transform ...
0
votes
0answers
11 views

Sequentia data with static data (gender, group, Major) in RNN model (LSTM)

I have a sequential dataset and I built an LSTM model using Keras for prediction accuracy. I would like to add static data ( i.e. gender, blood group, major) to the sequential data. I found a paper (...
1
vote
0answers
18 views

Structure of a multilayered LSTM neural network?

I implemented a LSTM neural network in Pytorch. It worked but I want to know if it worked the way I guessed how it worked. Say there's a 2-layer LSTM network with 10 units in each layer. The inputs ...
0
votes
1answer
15 views

Where can I find the original paper that established RNN?

I was able to find the original paper on LSTM (https://www.bioinf.jku.at/publications/older/2604.pdf). But I have had no luck with finding the original paper on RNN. Can someone please give me a link ...
2
votes
1answer
55 views

Why would giving my AI more data make it perform worse?

So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample. Next I tried to get it to generate limericks using these 100,000 limericks. It generated ...
0
votes
0answers
54 views

Deep learning with a lot of training data

I am building a bidirectional LSTM to do a sequential text-tagging task (particularly, automatic punctuation). Usually, the training is done in iterations, where in each iteration, the entire training ...
0
votes
1answer
59 views

LSTM RNN structure

I am currently looking into LSTMs. I found this nice blog post, which is already very helpful, but still, there are things I don't understand, mostly because of the collapsed layers. The input $X_t$,...
3
votes
1answer
56 views

How does LSTM in deep reinforcement learning differ from experience replay?

In the paper Deep Recurrent Q-Learning for Partially Observable MDPs, the author processed the Atari game frames with an LSTM layer at the end. My questions are: How does this method differ from the ...
0
votes
0answers
33 views

Speech recognition/ Voice Recognition: How to use NLP after a speech-to-text to improve accuracy?

I am planning to use Mozilla DeepSpeech for the project. For the use case at hand, I'm thinking if there's a possibility of using any NLP engine after the speech-to-text to improve its efficiency. If ...
3
votes
1answer
42 views

Using different timesteps for features and target value

I would like to know whether it's wrong; when working with time series data; to use daily prices as features and the price after 3 days as target. Is this correct or should I use the next-day price ...
4
votes
1answer
61 views

What would be the best approach to teach an AI to learn how to “sing” along a beat?

I have heard and read about HyperGAN, LSTM and a few other techniques, but I have a hard time piecing the overall concept together. End Goal Being able to input an instrumental and get an output of ...
1
vote
0answers
39 views

Using CNN LSTMs for prediction of images from image series

I have the following setup for a prediction task: I want to predict entire pictures from previously given pictures. In my case, only 2 pixels in every frame are neither black nor white, they are some ...
1
vote
0answers
24 views

Generating time series for doing time-series forecasting with LSTM

I have a .db file with columns as described below. This data has been collected by a software which monitors the file usage in a filesystem or in other words generates metadata about all the files in ...
1
vote
0answers
20 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
1
vote
1answer
86 views

Backward Pass for LSTMs

TL;DR I am currently trying to understand the mathematics in Ger's paper Long Short-Term Memory in Recurrent Neural Networks. I have found the document clear and readable so far. On pg. 21 of the ...
3
votes
2answers
410 views

Can LSTM Nets be speed up by GPU?

I am training LSTM Nets with Keras on a small mobile GPU. The speed on GPU is slower then on CPU. I found some articles that say that it is hard to train LSTMs (RNNs) on GPUs because the training ...
1
vote
0answers
19 views

How to visualize/interpret text prediction model results?

I am using LSTM model to predict the next xml markup from an input seed. I have trained my model on 1500 xml files. Each xml file is generated randomly. I am wondering if there is a way to visualize ...
3
votes
1answer
99 views

Structure of LSTM RNNs

I have some very basic questions here. This is probably because I didn't read the relevant documents closely enough. If I used some terminology incorrectly, please point them out. Thank you! For ...
1
vote
0answers
57 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
3
votes
2answers
36 views

How should my output layer activate for a sequence to sequence LSTM model with word2vec output

I'm having trouble grasping how to output word vectors from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot ...
0
votes
0answers
110 views

Load forecasting using LSTM in Matlab 2018a

I am using MATLAB 2018a, and looking for an explanation, or pointers, on how can I modify the matlab example for "sequence to sequence regression" to use it with "double" type predictor data array ...
3
votes
1answer
109 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
1
vote
0answers
41 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
5
votes
4answers
141 views

Use Machine/Deep Learning to Guess a String

I want to be able to input a block of text and then have it guess a string within a predefined range (i.e. a string that starts with three letters and ends with five numbers like "XXX12345", etc). ...
1
vote
2answers
55 views

Time Series: LSTM or Augmented Vector Space?

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors. There's two that I'm considering. An LSTM or ...
1
vote
0answers
29 views

Learning from events. Supervised, Unsupervised or MDP?

I have a large set of simulation logs for a market simulation of which I want to learn from. The market includes: customers products (subscriptions) The customers choose products and then stick with ...
3
votes
1answer
199 views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
0
votes
0answers
414 views

R-CNN LSTM (using Detectron) for image caption task

I am new to deep learning. I have run the caffe2 tutorials to train the LeNet for Mnist with my dataset and also the LSTM tutorial. Initially I was thinking of combining both systems to get a CNN ...
3
votes
1answer
407 views

What is the difference between ConvLSTM and CNN LSTM?

What will be the difference when used for video classification? Will they yield different results or are they the same fundamentally?
2
votes
1answer
95 views

How are LSTM's trained for text generation?

I've seen some articles about text generation using LSTMs (or GRUs) for text generation. Basically it seems you train them by folding them out, and putting a letter in each input. But say you trained ...
2
votes
0answers
58 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
7
votes
5answers
5k views

Using Machine/Deep learning for guessing Pseudo Random generator

Is it possible to feed a neural network, the output from a random number generator and expect it learn the hashing/generator function. So that it can predict what will be the next generated number? ...
6
votes
2answers
127 views

Shortening the development time of a neural network

I am developing an LSTM for sequence tagging. During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. After ...
2
votes
0answers
44 views

seq2seq vector to letters model

I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
2
votes
1answer
161 views

The state of a recurrent neural network

I am Reading "Supervised Sequence Labelling with Recurrent Neural Networks" written by Alex Graves to try to understand LSTM networks and I am a bit confused about the equations. Specifically, what I ...
1
vote
1answer
48 views

Multi-param LSTM input

Lets say you install your LSTM machine on a road between London and Oxford. And it makes observations. A car with 3 people inside drives past it in one direction 21 sec after previously observed car (...
10
votes
4answers
14k views

How to select number of hidden layers and number of memory cells in LSTM

As I am learning about LSTMs, but also about neural networks in general, I am trying to find some existing research on how to select the number of hidden layers and the size of these. Is there an ...
2
votes
0answers
437 views

Text Categorization using LSTM, word embeddings

I am currently reading the paper "Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings", and I have some difficulties understanding some of their simplifications to an ...
9
votes
2answers
4k views

How to train a chatbot

I wanted to started experimenting with neural network and as a toy problem I wished to train one to chat, i.e. implement a chatting bot like cleverbot. Not that clever anyway. I looked around for ...
1
vote
2answers
412 views

In LSTM text generation can low amount of training data be compensated?

I'm playing with an LSTM to generate text. In particular, this one: https://raw.githubusercontent.com/fchollet/keras/master/examples/lstm_text_generation.py It works on quite a big demo text set ...