Questions tagged [gated-recurrent-unit]

For questions related to the gated recurrent unit (GRU), a modification and simplification of the LSTM unit, which is a more sophisticated unit (with respect to the standard one) of a recurrent neural network (RNN). An RNN that uses GRU units is often called a GRU network. GRUs were introduced in the paper "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation" (2014) by Kyunghyun Cho et al.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

Finding "look_back" & "look_ahead" hyper-parameters for Seq2Seq models

For Seq2Seq deep learning architectures, viz., LSTM/GRU and multivariate, multistep time series forecasting, it is important to convert the data to a 3D dimension: (batch_size, look_back, ...
user avatar
  • 183
0 votes
0 answers
53 views

How to visualize the input and output of the GRU cell?

GRU belongs to the family of recurrent neural networks. This family of neural networks works on sequence data. But, it is taking time for me to understand the differences between sequence length and ...
user avatar
  • 3,131
0 votes
1 answer
33 views

Why does validation accuracy stop rising so soon?

I have implemented a GRU to deal with youtube comment data. I am a bit confused about why the validation score seems to even out around 70% and then keeps rising, this doesn't look like overfitting ...
user avatar
  • 1
0 votes
1 answer
65 views

Is my intuition about RNN wrong?

Until today, my intuition about RNN (LSTM/GRU) was that this is some kind of NN that can remember previous inputs. Consider a task where you need to predict 0 if the previous input was 1. For example: ...
user avatar
  • 109
0 votes
0 answers
90 views

Multiple GRU layers to improve a text generation

I am using the model in this colab https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/text_generation.ipynb#scrollTo=AM2Uma_-yVIq for Shakespeare like text generation. ...
user avatar
  • 101
0 votes
0 answers
88 views

What do RNN, LSTM, and GRU layers do in Tensorflow?

I have gone through some theoretical introductions of RNN and LSTM, which do not contain any code, and they describe in fair detail what the cells do, how they apply operations like forget, sigmoid, ...
user avatar
  • 101
0 votes
1 answer
43 views

What is it about sigmoid activations in particular that allows for the keeping and forgetting of past information from different time scales?

My understanding is that normal recurrent neural networks (RNNs) are not good at keeping past information from different time scales. Furthermore, my understanding is that Gated RNNs, such as Long ...
user avatar
2 votes
0 answers
40 views

Can One-Hot Vectors be used as Inputs for Recurrent Neural Networks?

When using an RNN to encode a sentence, one normally takes each word, passes it through an embedding layer, and then uses the dense embedding as the input into the RNN. Lets say instead of using dense ...
user avatar
1 vote
0 answers
732 views

What is the computational complexity in terms of Big-O notation of a Gated Recurrent Unit Neural network?

I have been digging up of articles across the internet in context of computational complexity of GRU. Interestingly, I came across this article, http://cse.iitkgp.ac.in/~psraja/FNNs%20,RNNs%20,LSTM%...
user avatar
3 votes
0 answers
204 views

What is the time complexity for training a gated recurrent unit (GRU) neural network using back-propagation through time?

Let us assume we have a GRU network containing $H$ layers to process a training dataset with $K$ tuples, $I$ features, and $H_i$ nodes in each layer. I have a pretty basic idea how the complexity of ...
user avatar
2 votes
0 answers
43 views

Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
user avatar
  • 31
6 votes
1 answer
745 views

What's the difference between LSTM and GRU?

I have been reading about LSTMs and GRUs, which are recurrent neural networks (RNNs). The difference between the two is the number and specific type of gates that they have. The GRU has an update gate,...
user avatar
  • 1,193
1 vote
1 answer
32 views

Inner working of Bidirectional RNNs

I'm trying to understand how Bidirectional RNNs work. Specifically, I want to know whether a single cell is used with different states, or two different cells are used, each having independent ...
user avatar
4 votes
0 answers
198 views

RNN models displays upper limit on predictions

I have trained a RNN, GRU, and LSTM on the same dataset, and looking at their respective predictions I have observed, that they all display an upper limit on the value they can predict. I have ...
user avatar
2 votes
1 answer
1k views

How do I choose the size of the hidden state of a GRU?

I'm trying to understand how the size of the hidden state affects the GRU. For example, suppose I want to make a GRU count. I'm gonna feed it with three numbers, and I expect it to predict the ...
user avatar
  • 1,098
5 votes
2 answers
198 views

Why are GRU and LSTM better than standard RNNs?

It seems that older RNNs have a limitation for their use cases and have been outperformed by other recurrent architectures, such as the LSTM and GRU.
user avatar