Skip to main content

Questions tagged [gated-recurrent-unit]

For questions related to the gated recurrent unit (GRU), a modification and simplification of the LSTM unit, which is a more sophisticated unit (with respect to the standard one) of a recurrent neural network (RNN). An RNN that uses GRU units is often called a GRU network. GRUs were introduced in the paper "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation" (2014) by Kyunghyun Cho et al.

11 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4 votes
0 answers
297 views

RNN models displays upper limit on predictions

I have trained a RNN, GRU, and LSTM on the same dataset, and looking at their respective predictions I have observed, that they all display an upper limit on the value they can predict. I have ...
Kornephoros's user avatar
3 votes
0 answers
445 views

What is the time complexity for training a gated recurrent unit (GRU) neural network using back-propagation through time?

Let us assume we have a GRU network containing $H$ layers to process a training dataset with $K$ tuples, $I$ features, and $H_i$ nodes in each layer. I have a pretty basic idea how the complexity of ...
rahul tomar's user avatar
2 votes
0 answers
121 views

Can One-Hot Vectors be used as Inputs for Recurrent Neural Networks?

When using an RNN to encode a sentence, one normally takes each word, passes it through an embedding layer, and then uses the dense embedding as the input into the RNN. Lets say instead of using dense ...
chessprogrammer's user avatar
2 votes
0 answers
47 views

Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
DoKi's user avatar
  • 31
1 vote
0 answers
45 views

The model's accuracy becomes suddenly so unreasonably good at beginning of the training process. I need an explaination

I am practicing machine translation using seq2seq model (more specifically with GRU/LSTM units). The following is my first model: This model first archived about 0.03 accuracy score and gradually ...
Đạt Trần's user avatar
1 vote
0 answers
2k views

What is the computational complexity in terms of Big-O notation of a Gated Recurrent Unit Neural network?

I have been digging up of articles across the internet in context of computational complexity of GRU. Interestingly, I came across this article, http://cse.iitkgp.ac.in/~psraja/FNNs%20,RNNs%20,LSTM%...
rahul tomar's user avatar
0 votes
0 answers
8 views

Is it possible to apply transfer learning between Temporal Fusion Transformer and sequential architecture LSTM and GRU

If TFT is a pretrained model, is it possible to transfer the weights to sequential neural network models like LSTM,BILSTM and GRU.
Santhana Lakshmi's user avatar
0 votes
0 answers
28 views

"The single scalar stored by an LSTM or GRU memory cell" - Deep learning book

I am reading Deep Learning by Goodfellow, Bengio, and Courville, and on page 413, they discuss how to store information using a framework such as a neural Turing machine. Quote: Neural networks excel ...
Kaira's user avatar
  • 101
0 votes
0 answers
19 views

Why use auto-regressive models for time-series?

This is a naive question... But I realized that auto-regressive predictions can be inherently unstable due to previous prediction error monotonically accumulating in the inputs: $M(h_{t-n},...,h_{t-m},...
profPlum's user avatar
  • 434
0 votes
0 answers
215 views

How to visualize the input and output of the GRU cell?

GRU belongs to the family of recurrent neural networks. This family of neural networks works on sequence data. But, it is taking time for me to understand the differences between sequence length and ...
hanugm's user avatar
  • 3,890
0 votes
0 answers
159 views

What do RNN, LSTM, and GRU layers do in Tensorflow?

I have gone through some theoretical introductions of RNN and LSTM, which do not contain any code, and they describe in fair detail what the cells do, how they apply operations like forget, sigmoid, ...
Della's user avatar
  • 111