Questions tagged [recurrent-neural-networks]

For questions related to recurrent neural networks (RNNs), artificial neural networks that contain backward or self-connections, as opposed to just having forward connections, like in a feed-forward neural network. An RNN can be trained using back-propagation through time, such that these backward connections "memorize" previously seen inputs. Consequentially, RNNs are well suited to sequence prediction and similar tasks.

Filter by
Sorted by
Tagged with
34
votes
4answers
53k views

How to select number of hidden layers and number of memory cells in an LSTM?

I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN. Is there an article where this problem is being investigated, i.e., ...
12
votes
2answers
2k views

What is a Recurrent Neural Network?

Surprisingly this wasn't asked before - at least I didn't find anything besides some vaguely related questions. So, what is a recurrent neural network, and what are their advantages over regular NNs?
3
votes
2answers
309 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released this paper in 2012: http://cs229.stanford.edu/proj2012/BernalFokPidaparthi-FinancialMarketTimeSeriesPredictionwithRecurrentNeural.pdf It goes on to ...
10
votes
5answers
6k views

What is the fundamental difference between CNN and RNN?

What is the fundamental difference between convolutional neural networks and recurrent neural networks? Where are they applied?
8
votes
2answers
750 views

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
7
votes
2answers
4k views

Where can I find the original paper that introduced RNNs?

I was able to find the original paper on LSTM, I was not able to find the paper that introduced "vanilla" RNNs. Where can I find it?
4
votes
3answers
109 views

Can we optimize an optimization algorithm?

In this answer to the question Is an optimization algorithm equivalent to a neural network?, the author stated that, in theory, there is some recurrent neural network that implements a given ...
2
votes
2answers
360 views

What are some examples of LSTM architectures?

I've been doing some class assignments recently on building various neural networks. For convolutional networks, there are several well-known architectures such as LeNet, VGG etc. Such "classic" ...
1
vote
1answer
132 views

Do you need to store prevous values of weights and layers on recurrent layer while BPTT?

The Back propagation through time on recurrent layer is defined similar to normal one, means somethin like ...
5
votes
2answers
3k views

Spam Detection using Recurrent Neural Networks

I am working on this code for spam detection using recurrent neural networks. Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a ...
2
votes
2answers
72 views

Are there neural networks that accept graphs or trees as inputs?

As far I know, the RNN accepts a sequence as input and can produce as a sequence as output. Are there neural networks that accept graphs or trees as inputs, so that to represent the relationships ...
2
votes
0answers
46 views

What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?

I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
1
vote
1answer
62 views

What is teacher forcing?

In the paper Neural Programmer-Interpreters, the authors use the teacher forcing technique, but what exactly is it?
1
vote
0answers
24 views

How to derive asymptotic run-time of seq2seq RNN? [duplicate]

I want to know the computational complexity of a sequence to sequence RNN in terms of the input and output length. I was wondering how I might go about deriving this in both training and inference.
0
votes
0answers
50 views

Why is the value range of my LSTM model's prediction different from my test labels?

I am using LSTM to do time-series anomaly detection. The data is an hourly sensor input across multiple years (i.e. the global_active_energy attribute of the dataset from https://www.kaggle.com/uciml/...
-5
votes
2answers
410 views

What are the approaches to predict sequence of π numbers? [closed]

Given list of fixed numbers from a mathematical constant such as Pi, is it is possible to train AI to attempt to predict the next numbers? Which AI or neural network would be more suitable for this ...