Questions tagged [recurrent-neural-networks]

Use for questions about Recurrent Neural Networks

51 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
30 views

dimensions of hidden layer and cell state layer in LSTM

I was following some examples to get familiar with tensorflow LSTM related api, but noticed that all LSTM initialization functions require only num_units parameter which denotes number of hidden units ...
3
votes
0answers
32 views

How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?

It seems that stacking LSTM layers can be beneficial for some problem settings in order to learn higher levels of abstraction of temporal relationships in the data. There is already some discussion on ...
3
votes
0answers
94 views

What is the intuition behind the calculation of the similarity between encoder and decoder states?

Suppose that we are doing machine translation. We have a conditional language model with attention where we are are trying to predict a sequence $y_1, y_2, \dots, y_J$ from $x_1, x_2, \dots x_I$: $$P(...
3
votes
0answers
115 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
2
votes
1answer
54 views

Issue at training simple RNN for word generation

After completing Coursera course from Andrew Ng, I wanted to implement again simple RNN for generating dinosaurs name based on a text file containing around 800 dinosaurs name. This is done with ...
2
votes
1answer
32 views

How are the observations stored in the RNN that encodes the state?

I am a bit confused about observations in RL systems which use RNN to encode the state. I read a few papers like this and this. If I were to use a sequence of raw observations (or features) as an ...
2
votes
0answers
40 views

How can my Neural Network categorize message strings?

Abstract I wish to design a neural network that will categorize messages based on criteria I have predefined. It should feature the ability to be proactively trained as it continues its lifecycle. ...
2
votes
1answer
186 views

Additive Attention in Convolutional Networks

Attention has been used widely in recurrent networks to weight feature representations learned by the model. This is not a trivial task since recurrent networks have a hidden state that captures ...
2
votes
0answers
85 views

Why are all the actions converging to the same index?

I am using PPO with an LSTM agent. My agent is performing 10 actions for each episode, one action is corresponding to one LSTM timestep and the action space is discrete. I have only one reward per ...
2
votes
0answers
95 views

Update of weights in Recurrent Neural Network through back propagation

How does Recurrent Neural Network updates its weights and bias through backpropagation? Is time taken into account while updating the weights of a RNN using Backpropagation through time(BPTT)?"
2
votes
0answers
91 views

Combine two embeddding inputs to increase more performance in LSTM model

The situation I encountered here is that I have two inputs(for instance, image embedding, etc.) into the first lstm of a series of lstms to predict the next word to generate sentence(from the second ...
2
votes
0answers
34 views

Detecting symmetry in small images with RNN

My network works on 32x32 normalized (translationally) but noisy images. Its task it to determine whether image has simple symmetry (horizontal/vertical). It needs to be reasonably robust to rotation (...
2
votes
0answers
44 views

Recommendations on which architecture to use to guess appointment

I'm currently developping an application which allows psychologists to manage their schedule and budget. As a proof of concept, I would like to create an intelligent appointment service. There can be ...
2
votes
0answers
49 views

Are gradients of weights in RNNs dependent on the gradient of every neuron in that layer?

I am writing my own recurrent neural network in Java to understand the inner workings better. While working through the math, I found that in timesteps later than 2 the gradient of weight w of neuron ...
2
votes
0answers
63 views

seq2seq vector to letters model

I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
2
votes
0answers
140 views

Preprocessing of training dataset for machine learning

I'm developing a log analyzer to predict and find errors in an equipment. Each logged data contains the following format: ...
2
votes
2answers
242 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released this paper in 2012: http://cs229.stanford.edu/proj2012/BernalFokPidaparthi-FinancialMarketTimeSeriesPredictionwithRecurrentNeural.pdf It goes on to ...
1
vote
0answers
23 views

Is there an standard algorithm for giving options from an RNN?

You can feed books to an RNN and it learn how to produce text. What I'm interested in is an algorithm that, given say 20 letters it suggest, say the best 10 options for the next 10 letters. So for ...
1
vote
0answers
18 views

Literature on Sequence Regresssion

I have some rated time-sequential data and I would like to test if an ANN can learn a correlation between my measurements and ratings. I suspect I could just try a CNN where 1 Dimension is time or an ...
1
vote
0answers
14 views

A gated neural network for internal thought?

I have an idea for an RNN which has no separate internal memory state only an output. But there is a gate in which tells the neural network whether the output will be acted out in the physical world ...
1
vote
0answers
38 views

RNN weights when varying the input size

I have a time-varying input size vector for a RNN. However, I am facing some difficulties understanding how to deal with my network weights when the input changes. Say we have a set of natural ...
1
vote
0answers
70 views

How does bidirectional encoding allow the predicted word to indirectly “see itself”?

Before the release of BERT we used to say that it is not possible to train bidirectional models by simply conditioning each word on its previous and next words, since this would allow the word that’s ...
1
vote
0answers
38 views

What is the feasible neural network structure that can learn to identify types of trajectory of moving dots?

I have multiple image sequences, each of which contains an animation of two moving dots. The trajectory of the dots in a sequence is always cyclic (not necessarily circular). There are two types of ...
1
vote
2answers
75 views

Which algorithm should I use to map an input sentence to an output sentence?

I am new to NLP realm. If you have an input text "The price of orange has increased" and output text "Increase the production of orange". Can we make our RNN model to predict the output text? Or what ...
1
vote
0answers
24 views

Why validation performance is unstable for my LSTM based model (labelling problems)?

I have trained a recurrent neural network based on 1 stack of LSTM cells. I use it to solve a classification problem. The RNN cell has 48 hidden states. The output of the last unfolded LSTM cell is ...
1
vote
0answers
37 views

Predicting sine using LSTM: Small output range and delayed output?

I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict $0.5*sin(t + N)$ given $0.5*sin(t)$ as an exercise. I have tweaked the model, changing the output layer ...
1
vote
0answers
20 views

Under What Conditions Will a Hopfield Network Tend to Converge to Spurious States?

I'm relatively new to neural networks, and I've been trying to program my own Hopfield network. I got it to the point where it can reliably reproduce a single pattern from a completely scrambled ...
1
vote
0answers
191 views

Number of nodes in hidden layer for LSTM

I am starting to learn LSTM by understanding how it is used for creating a char-RNN and had a fundamental question. Does the number of nodes in the hidden layer need to be the same as that of the ...
1
vote
0answers
36 views

Train a recurrent neural network by concatenating time series. Is it safe?

As the title says, I want to train a Jordan network (i.e. a particular kind of recurrent neural network) using a certain number of time series. Let's say that $x_1, x_2, \ldots x_N$ are $N$ input ...
1
vote
0answers
42 views

How recurrent neural network work when predict many days?

I use recurrent neural network, RNNs have to get input one value per step and it will show one value output. If I have daily sale demand time series data. I want to predict sale demand for three ...
1
vote
0answers
132 views

Deep learning model (LSTM) with temporal and non temporal attributes

I'm working on a project to predict the usage of all the files in a filesystem in near future based on the metadata of the file system for past 6 months. I've got the following attributes about the ...
1
vote
0answers
53 views

How to adapt RNNs to variable frequency / framerate of inputs?

Say I have an application where the frequency of the input is known but can vary widely across sequences. For example, they may be audio recordings acquired at different frequency, or videos that come ...
1
vote
0answers
26 views

Whats advantages does a Loop Network have over a Feed Forward Network?

I am interested to see what advantages a Loop Network (Feed Forward Network that takes its output as input, I think it's called an RNN, not sure). The only result I found was that it was extremely ...
1
vote
0answers
79 views

Best practices to classify recurring patterns using an LSTM or GRU

I'm working with acoustic data (filterbank features) and I want to build a neural network to detect claps using an LSTM (or a GRU) with a binary output (present/abscent), and I'm wondering about how I ...
1
vote
0answers
143 views

Trajectory classification using RNN

The problem: I want to classify a trajectory if it has some properties, for example I want to create a simple 0/1 classifier for circular trajectories. If a target is moving in a circular trajectory ...
1
vote
0answers
30 views

How to train a recurrent neural network with multiple series

I am new to neural networks. I am trying to model the run-off vs. time in a water channel after a storm event given that I know the permeability of the material in the channel, total precipitation, ...
1
vote
0answers
249 views

Tensorflow: Can't overfit training data with batch size > 1

I coded a small RNN network with Tensorflow to return the total energy consumption given some parameters. There seem to be a problem in my code. It can't overfit the training data when I use a batch ...
1
vote
0answers
23 views

Can one use a trained recurrent neural network to find the most probable sequence of a set?

Let's say I train an RNN on the sequence of characters in War and Peace. After doing so is there a way I can take an arbitrary set of letters, e.g. HNTASK, and have it (ideally) tell me that the most ...
1
vote
0answers
95 views

Infer dependent variables to produce output aligned to trained data

Hypothetical example, say I wanted: P(gender,ethnicity|age,hair); so that the input would aligned to a trained dataset of: ...
0
votes
0answers
5 views

Why word embedding such as word2vec is not used as the output layer of a seq2seq decoder?

It would make sense to make the decoder predict a smaller embedding vector instead of softmax over a large dictionary. The word having the most cosine similarity with the output embedding could be ...
0
votes
0answers
17 views

Better versions of LSTM

I know recurrent neural networks for example LSTM is one of them. After LSTM different variants of LSTM have come for example GRU. I don't know about newer RNN like LSTM and GRU, so I decided to ask ...
0
votes
0answers
44 views

Why doesnt my lstm model for time series prediction improve after certain level of performance?

I created an lstm model which predicts multioutput sequeances. It takes variable length sequences as input. These sequences are padded with zero to obtain equal length. Note that the time series are ...
0
votes
0answers
17 views

Dynamic frames processing with CNN LSTM combination or otherwise

I have a unique implementation where I have to process videos with dynamic frame rates (that is the number of frames is different for each video in a batch). I am stacking all the frames in a single ...
0
votes
1answer
59 views

How can I stabilise a recurrent neural network used for binary classification?

I’m looking for some help with my neural network. I’m working on a binary classification on a recurrent neural network that predicts stock movements (up and down) Let’s say I’m studying Eur/Usd, I’m ...
0
votes
0answers
65 views

How Seq2Seq with Bidirectional RNN works?

First of all the scope of the question is as follows - we have Sequence2Sequence architecture with: Decoder: Bidirectional LSTM Encoder: regular (single directional) LSTM What I know: When you ...
0
votes
0answers
15 views

Using the cloud service to trasform a picture using a neural algorithm?

yesterday I tried to transform a picture in the artistic style using CNNs based on A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge using a recent Torch ...
0
votes
0answers
11 views

How is the length of an input sequence related to the structure of an RNN?

My question is only with regards to the feedforward part of an RNN. I am following these steps. I am working on prediction of a time series. The time series is a toy model generated by me. It is ...
0
votes
0answers
31 views

Which neural network to use for mapping a vector of size m to a vector of size n, where n >> m?

I am trying to solve a mapping problem on a grid (100x100) where I have few points, say 10, where I know the values of a tensor $\boldsymbol{M}$. I have a scalar field, $v$, which is related to the ...
0
votes
1answer
73 views

how to predict the stock of food items using ai

Actually, I want to make an AI model which tells the seller about the maintaining stock of food items as a parameter of time and eventually learns by itself with the customer buying data (problem - ...
-1
votes
1answer
48 views

Teaching a NN to manipulate pseudoRNG over a long time scale?

For speedrunning purposes, I am trying to train a neural network to identify human-executable ways to manipulate pseudo-RNG (in Pokemon Red, for the interested). The game runs at sixty frames per ...