Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [gated-recurrent-unit]

For questions related to the gated recurrent unit (GRU), a modification and simplification of the LSTM unit, which is a more sophisticated unit (with respect to the standard one) of a recurrent neural network (RNN). An RNN that uses GRU units is often called a GRU network. GRUs were introduced in the paper "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation" (2014) by Kyunghyun Cho et al.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Is it possible to apply transfer learning between Temporal Fusion Transformer and sequential architecture LSTM and GRU

If TFT is a pretrained model, is it possible to transfer the weights to sequential neural network models like LSTM,BILSTM and GRU.
Santhana Lakshmi's user avatar
0 votes
0 answers
25 views

"The single scalar stored by an LSTM or GRU memory cell" - Deep learning book

I am reading Deep Learning by Goodfellow, Bengio, and Courville, and on page 413, they discuss how to store information using a framework such as a neural Turing machine. Quote: Neural networks excel ...
Kaira's user avatar
  • 101
0 votes
0 answers
18 views

Why use auto-regressive models for time-series?

This is a naive question... But I realized that auto-regressive predictions can be inherently unstable due to previous prediction error monotonically accumulating in the inputs: $M(h_{t-n},...,h_{t-m},...
profPlum's user avatar
  • 424
0 votes
0 answers
55 views

Feature extraction from log-mel spectrograms using CNNs

I am currently working on an ASR-related project in which I would like to combine Convolutional Neural Network with GRU Network and CTC loss function. The idea is to use the CNN to extract ...
conyai's user avatar
  • 1
1 vote
0 answers
45 views

The model's accuracy becomes suddenly so unreasonably good at beginning of the training process. I need an explaination

I am practicing machine translation using seq2seq model (more specifically with GRU/LSTM units). The following is my first model: This model first archived about 0.03 accuracy score and gradually ...
Đạt Trần's user avatar
1 vote
1 answer
71 views

Which calculation to use for GRU

Im doing trying to implement GRU in my own Neural Network Library but when I did some research i stumbled on some inconsistencies. When calculating a cell there are as many legitimate resources which ...
Johannes K.'s user avatar
0 votes
0 answers
210 views

How to visualize the input and output of the GRU cell?

GRU belongs to the family of recurrent neural networks. This family of neural networks works on sequence data. But, it is taking time for me to understand the differences between sequence length and ...
hanugm's user avatar
  • 3,890
0 votes
1 answer
105 views

Why does validation accuracy stop rising so soon?

I have implemented a GRU to deal with youtube comment data. I am a bit confused about why the validation score seems to even out around 70% and then keeps rising, this doesn't look like overfitting ...
nibs's user avatar
  • 1
0 votes
1 answer
136 views

Is my intuition about RNN wrong?

Until today, my intuition about RNN (LSTM/GRU) was that this is some kind of NN that can remember previous inputs. Consider a task where you need to predict 0 if the previous input was 1. For example: ...
Artod's user avatar
  • 109
2 votes
1 answer
933 views

Multiple GRU layers to improve a text generation

I am using the model in this colab https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/text_generation.ipynb#scrollTo=AM2Uma_-yVIq for Shakespeare like text generation. ...
kiriloff's user avatar
  • 121
0 votes
0 answers
151 views

What do RNN, LSTM, and GRU layers do in Tensorflow?

I have gone through some theoretical introductions of RNN and LSTM, which do not contain any code, and they describe in fair detail what the cells do, how they apply operations like forget, sigmoid, ...
Della's user avatar
  • 111
0 votes
1 answer
55 views

What is it about sigmoid activations in particular that allows for the keeping and forgetting of past information from different time scales?

My understanding is that normal recurrent neural networks (RNNs) are not good at keeping past information from different time scales. Furthermore, my understanding is that Gated RNNs, such as Long ...
The Pointer's user avatar
2 votes
0 answers
114 views

Can One-Hot Vectors be used as Inputs for Recurrent Neural Networks?

When using an RNN to encode a sentence, one normally takes each word, passes it through an embedding layer, and then uses the dense embedding as the input into the RNN. Lets say instead of using dense ...
chessprogrammer's user avatar
1 vote
0 answers
2k views

What is the computational complexity in terms of Big-O notation of a Gated Recurrent Unit Neural network?

I have been digging up of articles across the internet in context of computational complexity of GRU. Interestingly, I came across this article, http://cse.iitkgp.ac.in/~psraja/FNNs%20,RNNs%20,LSTM%...
rahul tomar's user avatar
3 votes
0 answers
431 views

What is the time complexity for training a gated recurrent unit (GRU) neural network using back-propagation through time?

Let us assume we have a GRU network containing $H$ layers to process a training dataset with $K$ tuples, $I$ features, and $H_i$ nodes in each layer. I have a pretty basic idea how the complexity of ...
rahul tomar's user avatar
2 votes
0 answers
47 views

Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
DoKi's user avatar
  • 31
6 votes
1 answer
2k views

What's the difference between LSTM and GRU?

I have been reading about LSTMs and GRUs, which are recurrent neural networks (RNNs). The difference between the two is the number and specific type of gates that they have. The GRU has an update gate,...
Pluviophile's user avatar
  • 1,273
1 vote
1 answer
42 views

Inner working of Bidirectional RNNs

I'm trying to understand how Bidirectional RNNs work. Specifically, I want to know whether a single cell is used with different states, or two different cells are used, each having independent ...
Susmit Agrawal's user avatar
4 votes
0 answers
290 views

RNN models displays upper limit on predictions

I have trained a RNN, GRU, and LSTM on the same dataset, and looking at their respective predictions I have observed, that they all display an upper limit on the value they can predict. I have ...
Kornephoros's user avatar
3 votes
1 answer
3k views

How do I choose the size of the hidden state of a GRU?

I'm trying to understand how the size of the hidden state affects the GRU. For example, suppose I want to make a GRU count. I'm gonna feed it with three numbers, and I expect it to predict the ...
razvanc92's user avatar
  • 1,138
6 votes
2 answers
398 views

Why are GRU and LSTM better than standard RNNs?

It seems that older RNNs have a limitation for their use cases and have been outperformed by other recurrent architectures, such as the LSTM and GRU.
Deep Analytics's user avatar