I am writing my own recurrent neural network in Java to understand the inner workings better. While working through the math, I found that in timesteps later than 2 the gradient of weight w of neuron n depends on the gradients of all neurons at all timesteps before. A handwritten example is given, I tried to write as clearly as possible.
Could anyone verify this so I can finish my network? Am I missing a piece or is my premise wrong, as in the output of a neuron is s(Wx + Vh + b), where h is the last step of only neuron n?