Questions tagged [recurrent-neural-networks]
For questions related to recurrent neural networks (RNNs), artificial neural networks that contain backward or self-connections, as opposed to just having forward connections, like in a feed-forward neural network. An RNN can be trained using back-propagation through time, such that these backward connections "memorize" previously seen inputs. Consequentially, RNNs are well suited to sequence prediction and similar tasks.
19
questions
68
votes
4
answers
124k
views
How to select number of hidden layers and number of memory cells in an LSTM?
I am trying to find some existing research on how to select the number of hidden layers and the size of these of an LSTM-based RNN.
Is there an article where this problem is being investigated, i.e., ...
- 1,019
39
votes
2
answers
23k
views
How can Transformers handle arbitrary length input?
The transformer, introduced in the paper Attention Is All You Need, is a popular new neural network architecture that is commonly viewed as an alternative to recurrent neural networks, like LSTMs and ...
- 3,000
76
votes
4
answers
109k
views
Why does the transformer do better than RNN and LSTM in long-range context dependencies?
I am reading the article How Transformers Work where the author writes
Another problem with RNNs, and LSTMs, is that it’s hard to parallelize the work for processing sentences, since you have to ...
- 1,743
13
votes
5
answers
8k
views
What is the fundamental difference between CNN and RNN?
What is the fundamental difference between convolutional neural networks and recurrent neural networks? Where are they applied?
- 151
13
votes
3
answers
3k
views
What is a recurrent neural network?
Surprisingly, this wasn't asked before - at least I didn't find anything besides some vaguely related questions.
So, what is a recurrent neural network, and what are their advantages over regular (or ...
- 759
8
votes
1
answer
30k
views
What is the difference between LSTM and RNN?
What is the difference between LSTM and RNN? I know that RNN is a layer used in neural networks, but what exactly is an LSTM? Is it also a layer with the same characteristics?
- 83
5
votes
1
answer
509
views
What is the significance of this Stanford University "Financial Market Time Series Prediction with RNN's" paper?
Researchers at Stanford University released, in 2012, the paper Financial Market Time Series Prediction with Recurrent Neural Networks.
It goes on to discuss how they used echo state networks to ...
3
votes
1
answer
1k
views
What is teacher forcing?
In the paper Neural Programmer-Interpreters, the authors use the teacher forcing technique, but what exactly is it?
- 41.1k
21
votes
4
answers
32k
views
Where can I find the original paper that introduced RNNs?
I was able to find the original paper on LSTM, but I was not able to find the paper that introduced "vanilla" RNNs. Where can I find it?
- 311
4
votes
2
answers
1k
views
Are there neural networks that accept graphs or trees as inputs?
As far I know, the RNN accepts a sequence as input and can produce as a sequence as output.
Are there neural networks that accept graphs or trees as inputs, so that to represent the relationships ...
- 358
3
votes
2
answers
5k
views
Spam Detection using Recurrent Neural Networks [closed]
I am working on this code for spam detection using recurrent neural networks.
Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a ...
- 177
3
votes
0
answers
103
views
What is the difference between zero-padding and character-padding in Recurrent Neural Networks?
For RNN's to work efficiently, we vectorize the operations, which results in an input matrix of shape
(m, max_seq_len)
where m ...
- 31
2
votes
1
answer
4k
views
What are some examples of LSTM architectures?
I've been doing some class assignments recently on building various neural networks. For convolutional networks, there are several well-known architectures such as LeNet, VGG etc. Such "classic" ...
- 181
1
vote
1
answer
295
views
Do you need to store prevous values of weights and layers on recurrent layer while BPTT?
The Back propagation through time on recurrent layer is defined similar to normal one, means somethin like
...
- 358
1
vote
0
answers
78
views
Is the capability of RNN more than the capability of MLP?
Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named ...
- 3,960
1
vote
0
answers
137
views
Is a true RNN auto encoder possible with Keras/TF
I want to get some encodings for temporal data (with a highly varying number of timesteps).
The dataset is of the format: ...
- 141
0
votes
1
answer
1k
views
How to process data in a data stream for a LSTM
How can a data stream for a RNN (LSTM) be handled, when the stream contains data sets belonging to different prediction classes?
Training phase:
I have trained a LSTM to predict a class out of a ...
- 445
0
votes
1
answer
55
views
What is it about sigmoid activations in particular that allows for the keeping and forgetting of past information from different time scales?
My understanding is that normal recurrent neural networks (RNNs) are not good at keeping past information from different time scales. Furthermore, my understanding is that Gated RNNs, such as Long ...
- 603
-4
votes
2
answers
879
views
What are the approaches to predict sequence of $\pi$ numbers? [closed]
Given a list of fixed numbers from a mathematical constant, such as $\pi$, is it is possible to train AI to attempt to predict the next numbers of this constant?
Which AI or neural network would be ...
- 10.5k