Do you need to store prevous values of weights and layers on recurrent layer while BPTT?

Question

The Back propagation through time on recurrent layer is defined similar to normal one, means somethin like

self.deltas[x] = self.deltas[x+1].dot(self.weights[x].T) * self.layers[x] * (1- self.layers[x]) where

self.deltas[x+1] is error from prevous layer, self.weights[x] is weights map and self.layers[x](1- self.layers[x]) is bakwards activation of sigmoid function where self.layers[x] is vector of sigmoid. But while normal backpropagation the values are there, while BPTT i can not take the current self.layers[x] : i need the previous ones, right ?

So unlike normal BP, do i need extra store old weights and layers, for example in circular queue, and then apply the formula where self.deltas[x+1] is layer from next time ?

Not realy implementation, just basic understanding in order to can implement it.

Lets see the picture:

Here are : self.layers[0] = $x_{t+1}$, self.layers[1] = $h_{t+1}$ , self.layers[2] = $o_{t+1}$, in order to perform backprop $h_{t+1}$ -> $h_{t}$ -> $h_{t-1}$... I DO NEED to have layers $h_t$ ,$h_{t-1}$... and weights $v_{t+1}$, $v_t$... EXTRA stored in additional to the network $x_{t+1}$ -> $h_{t+1}$ -> $o_{t+1}$, right? Thats all the question.

And i do not need to store previous outputs $o[t, o_{t-1}, etc..]$, because backprop from them ot->ht, etc was already calculated.

Answer 1

One word answer for your question "Do you need to store previous values of weights and layers on recurrent layer while BPTT?" is YES

Let us go through the details.

For training an RNN using BPTT, we need gradients of error w.r.t all three parameters U, V, W

Notation of my explanation is different from notation in the figure of question. My notation is as below:

1. V - Hidden Layer - Output Layer (gradients of V are independent of previous time steps)
2. U - Input Layer - Hidden Layer (gradients of U are dependent on previous time steps)
3. W - Hidden Layer - Hidden Layer (gradients of W are also dependent on previous time steps)

And for calculating these gradients, we use chain rule of differentiation, the same rule that we used to calculate gradients in a fully connected neural network.

The gradient w.r.t V only depends on current time step (doesn't need any values from previous time step).

The gradients w.r.t U, W depends on current time step and also all previous time steps (so needs values from all time steps)

Basically, we need to back propagate gradients from current time step all the way to t=0.

How this back propagation is different from the back propagation we use in fully connected neural network is that, in fully connected neural network we don't have the concept of t and also we don't share any weights across layers. But, here we share weights across layers and time instants. So, gradients depend on all time instants.

Note: Be careful with notation difference between several articles. I followed slightly different notation than in the diagram in question.

Some links that will help you explore.

https://www.youtube.com/watch?v=RrB605Mbpic (clearly explains about gradients of all three U, V, W; but notation is different from diagram in question)

http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/enter link description here

http://ir.hit.edu.cn/~jguo/docs/notes/bptt.pdf

https://www.d2l.ai/chapter_recurrent-neural-networks/bptt.html

Remember, you should understand chain rule of partial derivative very clearly to do the derivation yourself and understand it.

Also, dont think BPTT is separate than BP. It is one and the same. Since neural network architecture in RNN includes time instants and sharing of weights across time instants, just using chain rule on this network makes back propagation also dependent on time and so is the name.

Hope it helps. Feedback is welcome.