I am working on an implementation of the back propagation algorithm. What I have implemented so far seems working but I can't be sure that the algorithm is well implemented, here is what I have noticed during training test of my network:
Specification of the implementation:
- A data set containing almost 100000 raw containing (3 variable as input, the sinus of the sum of those three variables as expected output).
- The network does have 7 layers, all the layers use the sigmoid activation function
When I run the back propagation training process:
- The minimum of costs of the error is found at the fourth iteration (The minimum cost of error is 140, is it normal? I was expecting much less than that)
- After the fourth iteration the costs of the error start increasing (I don't know if it is normal or not?)