Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [lstm]

For questions about LSTM (long-short term memory) networks.

0
votes
0answers
9 views

How my LSTM is making association between the columns?

I have created LSTM model using the following Tensorflow code: ...
0
votes
0answers
12 views

Using a combination of gradient boosting with LSTM for classification? [migrated]

I am presently using a LSTM model to classify high dimensional tabular data which is not text/images (dimensions 21392x1970). I also tried XGBoost (Gradient boosting) in Python separately for the same ...
0
votes
0answers
10 views

Predicting global horizontal irradiance using satellite images [migrated]

I have the aim to build a model to predict global horizontal irradiance (ghi) using satellite images and other features namely the day of the year and time of the day. For extracting the satellite ...
1
vote
0answers
17 views

How to train chat bot on infinite non-stationary data?

I have continual simulated data of million sentences of two simulated persons talking to each other in a room and I want to model one of the persons speech. Now, during this period things in the room ...
1
vote
0answers
30 views

Experiment shows that LSTM does worse than Random Forest… Why?

LSTM is supposed to be the right tool to capture path-dependency in time-series data. I decided to run a simple experiment (simulation) to assess the extent to which LSTM is better able to understand ...
-1
votes
0answers
10 views

Slow moving auto parts data (public) from a US automobile company: where to find it? [migrated]

I have found multiple papers which use this dataset (because it's apparently public) to set benchmarks in forecasting but they don't mention where it can be found. I have done some searching on my own ...
1
vote
1answer
43 views

Is my Neural Network program fully connected?

I have the following program for my neural network: ...
2
votes
0answers
27 views

How does a neural network output text box location data?

I'm interested in creating a convolutional neural network or LSTM to locate text in an image. I don't want to OCR the text yet, just find the text regions. Yes, I know Tesseract and other systems can ...
1
vote
0answers
15 views

Kera's (normal) LSTM uses the GPU?

I'm running Kera's LSTM (not CuDNNLSTM) but I notice my GPU is under load. I need recurrent dropout, so I can only stick with <...
1
vote
0answers
33 views

how to estimate which item is bought most using AI

Problem: Maintaining the products is a big task for a retailer. If we can estimate using AI to predict which products will sell the most, we can maintain sufficient stocks of the product without ...
-2
votes
0answers
46 views

Prediction model as helper for a DQN agent

Suppose I have a regression model that can make predictions on stock price movements for 10 steps ahead. The labels are ...
0
votes
1answer
19 views

LSTM language model not working

I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons ...
1
vote
0answers
61 views

Number of nodes in hidden layer for LSTM

I am starting to learn LSTM by understanding how it is used for creating a char-RNN and had a fundamental question. Does the number of nodes in the hidden layer need to be the same as that of the ...
1
vote
2answers
26 views

Is there bidirection sequence-to-sequence neural machine translation?

I have heard about bidirectional RNN LSTM units (endcoders-decoders), but my question is - is there bidirectional neural machine translation, that uses A->B weights for the translation in the opposite ...
1
vote
1answer
31 views

Neurological interpretation of LSTMs

I am searching for an Interpretation of LSTMs and recurrent neural Networks within Cognitive Neuroscience. Is there a mechanism in the human brain, that works analog to LSTMs? How does Long-term ...
0
votes
1answer
20 views

Reduce the effect of excessive zeros

I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I ...
1
vote
1answer
78 views

Most efficient neural network for human activity recognition

A paper from machinelearningmastery.com on human activity recognition states that 1D convolutional neural networks work the best on classification of human activities using data from accelometer. But, ...
1
vote
0answers
17 views

Learning Tree Paths when Given Vectors

The problem statement: Mapping from a vector space representation onto a tree structure. Possible solution: Given a word vector as input, produce a path in the tree from the root down to the node ...
1
vote
0answers
27 views

Price difference predictions curve almost vanished

With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well ...
1
vote
1answer
101 views

Price Movement Forecasting Issue

I am working on a project for price movement forecasting and I am stuck with poor quality predictions. At every time-step I am using an LSTM to predict the next 10 time-steps. The input is the ...
1
vote
0answers
21 views

How to wire up bi-directional LSTMs?

I am interested in creating the neural network architecture described in this recent paper. However, I am not sure how to get started. Does TFLearn or Keras allow us to wire the LSTM in such ways?
1
vote
0answers
20 views

If there are several computers on a subnet, can training time be reduced by distributing the work across them?

We have multiple computers and the ability to ssh between them. What are options using either Java, C/C++, JavaScript, or Python to distribute our learning tasks? We will be using DCNN, DQN, and LSTM ...
1
vote
0answers
24 views

How to understand marginal loglikelihood objective function as loss function (explanation of an article)?

I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned ...
2
votes
1answer
72 views

Difficulty understanding Keras LSTM fitting data

I'm try to train a RNN with a chunk of audio data, where X and Y are two audio channels loaded into numpy arrays. The objective is to experiment with different NN designs to train them to transform ...
1
vote
0answers
58 views

Structure of a multilayered LSTM neural network?

I implemented a LSTM neural network in Pytorch. It worked but I want to know if it worked the way I guessed how it worked. Say there's a 2-layer LSTM network with 10 units in each layer. The inputs ...
3
votes
1answer
186 views

Where can I find the original paper that introduced RNNs?

I was able to find the original paper on LSTM, I was not able to find the paper that introduced "vanilla" RNNs. Where can I find it?
2
votes
1answer
63 views

Why would giving my AI more data make it perform worse?

So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample. Next I tried to get it to generate limericks using these 100,000 limericks. It generated ...
0
votes
1answer
78 views

LSTM RNN structure

I am currently looking into LSTMs. I found this nice blog post, which is already very helpful, but still, there are things I don't understand, mostly because of the collapsed layers. The input $X_t$,...
4
votes
1answer
1k views

How does LSTM in deep reinforcement learning differ from experience replay?

In the paper Deep Recurrent Q-Learning for Partially Observable MDPs, the author processed the Atari game frames with an LSTM layer at the end. My questions are: How does this method differ from the ...
3
votes
1answer
44 views

Using different timesteps for features and target value

I would like to know whether it's wrong; when working with time series data; to use daily prices as features and the price after 3 days as target. Is this correct or should I use the next-day price ...
4
votes
1answer
67 views

What would be the best approach to teach an AI to learn how to “sing” along a beat?

I have heard and read about HyperGAN, LSTM and a few other techniques, but I have a hard time piecing the overall concept together. End Goal Being able to input an instrumental and get an output of ...
1
vote
0answers
51 views

Using CNN LSTMs for prediction of images from image series

I have the following setup for a prediction task: I want to predict entire pictures from previously given pictures. In my case, only 2 pixels in every frame are neither black nor white, they are some ...
1
vote
0answers
112 views

Generating time series for doing time-series forecasting with LSTM

I have a .db file with columns as described below. This data has been collected by a software which monitors the file usage in a filesystem or in other words generates metadata about all the files in ...
1
vote
0answers
23 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
1
vote
1answer
187 views

Backward Pass for LSTMs

TL;DR I am currently trying to understand the mathematics in Ger's paper Long Short-Term Memory in Recurrent Neural Networks. I have found the document clear and readable so far. On pg. 21 of the ...
5
votes
2answers
2k views

Can LSTM Nets be speed up by GPU?

I am training LSTM Nets with Keras on a small mobile GPU. The speed on GPU is slower then on CPU. I found some articles that say that it is hard to train LSTMs (RNNs) on GPUs because the training ...
1
vote
0answers
22 views

How to visualize/interpret text prediction model results?

I am using LSTM model to predict the next xml markup from an input seed. I have trained my model on 1500 xml files. Each xml file is generated randomly. I am wondering if there is a way to visualize ...
5
votes
2answers
375 views

Structure of LSTM RNNs

I have some very basic questions here. This is probably because I didn't read the relevant documents closely enough. If I used some terminology incorrectly, please point them out. Thank you! For ...
1
vote
0answers
100 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
3
votes
2answers
62 views

How should my output layer activate for a sequence to sequence LSTM model with word2vec output

I'm having trouble grasping how to output word vectors from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot ...
3
votes
1answer
209 views

Over- and underestimations of the lowest and highest values in LSTM network

I'm training a LSTM network with multiple inputs and several LSTM layers in order to setup a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the ...
1
vote
0answers
71 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
5
votes
4answers
226 views

Use Machine/Deep Learning to Guess a String

I want to be able to input a block of text and then have it guess a string within a predefined range (i.e. a string that starts with three letters and ends with five numbers like "XXX12345", etc). ...
1
vote
2answers
78 views

Time Series: LSTM or Augmented Vector Space?

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors. There's two that I'm considering. An LSTM or ...
1
vote
0answers
42 views

Learning from events. Supervised, Unsupervised or MDP?

I have a large set of simulation logs for a market simulation of which I want to learn from. The market includes: customers products (subscriptions) The customers choose products and then stick with ...
3
votes
1answer
463 views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
2
votes
1answer
668 views

What is the difference between ConvLSTM and CNN LSTM?

What will be the difference when used for video classification? Will they yield different results or are they the same fundamentally?
2
votes
1answer
118 views

How are LSTM's trained for text generation?

I've seen some articles about text generation using LSTMs (or GRUs) for text generation. Basically it seems you train them by folding them out, and putting a letter in each input. But say you trained ...
2
votes
0answers
78 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
8
votes
5answers
8k views

Using Machine/Deep learning for guessing Pseudo Random generator

Is it possible to feed a neural network, the output from a random number generator and expect it learn the hashing/generator function. So that it can predict what will be the next generated number? ...