Skip to main content

New answers tagged

0 votes

Why the cost/loss starts to increase for some iterations during the training phase?

After reading your comments, I assume you are doing full-batch training instead of mini-batch training. Therefore, stochasticity is not a factor in this part. Despite the training batch size, I couldn’...
Barışcan Kurtkaya's user avatar
1 vote
Accepted

Distinguishing between the fundamental structures of the convolutional neural network and the recurrent neural network: hierarchical vs sequential

My answer is based on these 2 papers. https://arxiv.org/pdf/2110.13985 and https://arxiv.org/pdf/2111.00396 and here is the tutorial video https://www.youtube.com/watch?v=EvQ3ncuriCM that explains ...
saleh's user avatar
  • 148
1 vote
Accepted

Input size for a RNN layer by modeling a dynamical system

This the the initial condition of the system, namely the first derivative of displacement. You need somehow to provide the initial internal state of RNN and this is the way this could be done with ...
Damir Tenishev's user avatar
0 votes

What is the exact purpose of input modulation gate in LSTMs?

It's purely a naming thing (see: the footnote on page 3, for example). Recall that on each time-step, the concatenated input + hidden state from the prev. time-step are incorporated into the cell-...
Alexander Wan's user avatar

Top 50 recent answers are included