As I understand the gradient should reflect how near the weights are to the optimal values. In this way i will expect that on the first epochs the gradients far from zero or at least not mostly zero and as we train the net the gradients will arrive to values nearest to zero. But it is not the case as you can see for example here (This image show gradients distribution on each epoch):
and here (This image show gradients for 5 layers after the first batch):
I've seen the same behavior in other simple nets. Can someone explain this?