Questions tagged [lstm]

For questions about LSTM (long-short term memory) networks.

Filter by
Sorted by
Tagged with
0
votes
1answer
24 views

Standard test for a CNN and LSTM?

What are the "standard" tests (or at least common ones) for CNN's and LSTM's? As an example, for a feed-forward neural net, a common test is the xor test. Is there a test like this for a CNN and LSTM?...
1
vote
0answers
10 views

How does the Bidirectional RNN works?

Could it be possible to reach a similar output via feeding a unidirectional network with the original data and the data played backwards?
2
votes
1answer
55 views

Why does an LSTM cycle on initialisation?

I initialised an LSTM with Xavier initialisation, although I've found this occurs for all initialisations I have tested. When initialised, if the LSTM is tested with a random input, it will get stuck ...
4
votes
1answer
30 views

dimensions of hidden layer and cell state layer in LSTM

I was following some examples to get familiar with tensorflow LSTM related api, but noticed that all LSTM initialization functions require only num_units parameter which denotes number of hidden units ...
2
votes
1answer
30 views

What sort of Neural Network is best suited to predicting a future purchase?

I have previously implemented a Neural Network with Back-Propagation that was able to learn Tic-tac-toe and could go pretty well at Connect-4. Now I'm trying to do a NN that can make a prediction. ...
0
votes
2answers
54 views

LSTM network doesn't converge, what should be changed? [closed]

I'm testing out TensorFlow LSTM layer text generation task, not classification task; but something is wrong with my code, it doesn't converge. What changes should be done? Source code: ...
0
votes
0answers
12 views

Make an LSTM model for each class separately

I have a dataset of some activities. The dataset contains the status of different sensors and the label of activity. T trained a model in Keras with the following architecture which models the ...
0
votes
1answer
29 views

How to map X to Y for TensorFlow RNN training data

Usually for DNN, I have the training data of matching X (2D) to Y (2D), for example, XOR data: X = [[0,0],[0,1],[1,0],[1,1]]; Y = [[0], [1], [1], [0] ]; ...
3
votes
1answer
23 views

Structure discrepancy of an LSTM?

I've found multiple depictions of how an LSTM cell operates. See 2 below: and Each of these images suggest the hidden state is utilised differently. On the top diagram, it is shown that the hidden ...
1
vote
2answers
84 views

What are some examples of LSTM architectures?

I've been doing some class assignments recently on building various neural networks. For convolutional networks, there are several well-known architectures such as LeNet, VGG etc. Such "classic" ...
0
votes
2answers
30 views

How does an LSTM output the correct dimensions for classes?

Take the below LSTM: input: 5x1 matrix hidden units: 256 output size (aka classes, 1 hot vector): 10x1 matrix It is my understanding that an LSTM of this size ...
1
vote
0answers
25 views

Initial LSTM hidden state and cell

If we use LSTMCell from torch: The initial hidden and cell layers should be CONSTANT (from the first time you run the program) and saved right? Like random seeds? ...
2
votes
0answers
18 views

How can I detect fast and slow motion in videos?

I'm trying to detect if a given video shot is fast or slow motion. Basically, I need to calculate a "video motion" score in a given video sequence, meaning how fast or slow motion the video is. For ...
0
votes
1answer
46 views

LSTM in reinforcement learning

Please tell me that is the LSTM network for the problem of reinforcement learning, as I explain to her what she will get the reward of a prediction, because the output will contain only actions? Well,...
0
votes
1answer
37 views

Preparation of input data

Tell me why my val_acc is always the same and how to solve this problem? I saw several topics on the Internet specifically on this problem but they did not help me (for example, use SGD with different ...
2
votes
0answers
39 views

Spike detection in time series using Artificial Neural Networks

I'm quite new in ANNs. I intend to use ANNs for predicting spike points in time series right before they happen. I've already used LSTM for another scenario, and I know that they can be used in ...
1
vote
0answers
44 views

How to use the LSTM layer in PPO architecture?

What is the best way of using the LSTM layer in PPO architecture? Should I use them in the first layer of both actor and critic, or use them just before the final layer of these networks? Should I ...
1
vote
1answer
61 views

LSTM text classifier shows unexpected cyclical pattern in loss

I'm training a text classifier in PyTorch and I'm experiencing an unexplainable cyclical pattern in the loss curve. The loss drops drastically at the beginning of each epoch and then starts rising ...
0
votes
0answers
44 views

Why doesnt my lstm model for time series prediction improve after certain level of performance?

I created an lstm model which predicts multioutput sequeances. It takes variable length sequences as input. These sequences are padded with zero to obtain equal length. Note that the time series are ...
0
votes
0answers
23 views

Custom optimizer and word-vector evaluator lstm

I’m using Keras LSTM layers and building a model that is trained off ethics text. I have a problem of often over fitting (the network basically remembers my input corpus as it is very small). I was ...
1
vote
0answers
29 views

Keras CRNN implementation with multiple input images

Hello I am trying to implement a CRNN with multiple input images (in my context it is 6 images) This is a regression problem and output is two real value. And for the CNN block I am thinking of using ...
0
votes
0answers
66 views

How Seq2Seq with Bidirectional RNN works?

First of all the scope of the question is as follows - we have Sequence2Sequence architecture with: Decoder: Bidirectional LSTM Encoder: regular (single directional) LSTM What I know: When you ...
1
vote
1answer
95 views

Why can we approximate the joint probability distribution using the output vector of an LSTM?

In the paper, Contextual String Embeddings for Sequence Labeling, the authors state that \begin{equation} P(x_{0:T}) = \prod_{t=0}^T P(x_t|x_{0:t-1}) \end{equation} They also state that, in the LSTM ...
4
votes
1answer
54 views

Transformer based decoding

Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder ...
2
votes
1answer
68 views

LSTM Architecture

There is plenty of literature describing LSTMs in a lot of detail and how to use them for multi-variate or uni-variate forecasting problems. What I couldn't find though, is any papers or discussions ...
0
votes
2answers
101 views

Time series RNN vs DNN

Understandably RNNs are very good at solving problems involving audio, video and text processing due to arbitrary input length of this sort of data. What I don't understand is why RNNs are also ...
-1
votes
1answer
39 views

Error using Keras clear_session while multiple calibration

I am working through the tutorial here. I just cleaned up the code for myself but other than that code is as in original tutorial. I am using the pollution data from the following link When I try to ...
1
vote
1answer
92 views

Why do we need multiple LSTM units in a layer?

What is the point of having multiple LSTM units in a single layer? Surely if we have a single unit it should be able to capture (remember) all the data anyway and using more units in the same layer ...
0
votes
1answer
21 views

Modifying LSTM to include forecast

I am looking at LSTM example here However, I am not sure how to modify the setup if I have forecast available (assuming perfect forecast) for TEMP: Temperature and PRES: Pressure at time t. i.e. <...
0
votes
0answers
85 views

Details of implementing an LSTM in Reinforcement Learning

I'm currently looking into the context of adding an LSTM to my PPO pytorch implementation. My plan is to add one LSTM layer right after the last convolutional layer. I'm wondering now whether it is ...
2
votes
1answer
127 views

Object IN/OUT counting using CNN+RNN

I am building a video analytics program for counting moving things in a video. I am detecting bicycles and nothing else. I run object detection using the SSD mobile-net model in all the frames and ...
0
votes
0answers
36 views

Unsupervised LSTM

I have a big amount of light curves (image below) and I am trying to label the points as signal or background (the signal appears usually periodically, several times, for a given light curve). However,...
0
votes
0answers
17 views

Benefits in using multiple LSTM layers?

I am working on a time series forecasting problem and I am in the process of choosing the optimum network structure. Currently I have a 200 cell LSTM layer fully connected to 100 neurons in an ...
1
vote
0answers
11 views

Training LSTM with class imbalance

I need to train an LSTM on some light curves, in order to find a signal (there are 2 classes signal and background). However the signal (data points corresponding to signal) is around 100 times less ...
3
votes
0answers
95 views

What can be considered a deep recurrent neural network?

In the paper Deep Recurrent Q-Learning for Partially Observable MDPs, the DRQN is described as DQN with the first post-convolutional fully-connected layer replaced by a recurrent LSTM. I have DQN ...
2
votes
0answers
40 views

RNN: Different test results on balanced and unbalanced data

I trained a recurrent neural network (if it matters - it contains three CuDNNLSTM cells and 3 Dense layers, Dropout = 0.2). The result of data preparation is one array of ~330.000 sequences. Each ...
3
votes
0answers
32 views

How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?

It seems that stacking LSTM layers can be beneficial for some problem settings in order to learn higher levels of abstraction of temporal relationships in the data. There is already some discussion on ...
1
vote
1answer
46 views

DQN Q-values are static

I am working on a DDQN with 5 LSTM layers and 3 actions as output and state space of 21 features. I am dividing the dataset into episodes of 720 timesteps, for each episode the agent acts greedily for ...
2
votes
0answers
18 views

Multi-field text input for LSTM

I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using ...
1
vote
0answers
24 views

Why validation performance is unstable for my LSTM based model (labelling problems)?

I have trained a recurrent neural network based on 1 stack of LSTM cells. I use it to solve a classification problem. The RNN cell has 48 hidden states. The output of the last unfolded LSTM cell is ...
1
vote
1answer
109 views

LSTM is not converging

I am writing my first LSTM network and I would really appreciate if someone can tell me if it is right (the loss seems to go down very slowly and before playing around with hyper parameters I want to ...
1
vote
0answers
37 views

Predicting sine using LSTM: Small output range and delayed output?

I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict $0.5*sin(t + N)$ given $0.5*sin(t)$ as an exercise. I have tweaked the model, changing the output layer ...
1
vote
0answers
18 views

How to train chat bot on infinite non-stationary data?

I have continual simulated data of million sentences of two simulated persons talking to each other in a room and I want to model one of the persons speech. Now, during this period things in the room ...
1
vote
0answers
218 views

Experiment shows that LSTM does worse than Random Forest… Why?

LSTM is supposed to be the right tool to capture path-dependency in time-series data. I decided to run a simple experiment (simulation) to assess the extent to which LSTM is better able to understand ...
1
vote
1answer
60 views

Is my Neural Network program fully connected?

I have the following program for my neural network: ...
2
votes
0answers
32 views

How does a neural network output text box location data?

I'm interested in creating a convolutional neural network or LSTM to locate text in an image. I don't want to OCR the text yet, just find the text regions. Yes, I know Tesseract and other systems can ...
1
vote
0answers
24 views

Kera's (normal) LSTM uses the GPU?

I'm running Kera's LSTM (not CuDNNLSTM) but I notice my GPU is under load. I need recurrent dropout, so I can only stick with <...
1
vote
0answers
35 views

how to estimate which item is bought most using AI

Problem: Maintaining the products is a big task for a retailer. If we can estimate using AI to predict which products will sell the most, we can maintain sufficient stocks of the product without ...
0
votes
1answer
120 views

LSTM language model not working

I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons ...
1
vote
0answers
191 views

Number of nodes in hidden layer for LSTM

I am starting to learn LSTM by understanding how it is used for creating a char-RNN and had a fundamental question. Does the number of nodes in the hidden layer need to be the same as that of the ...