For a graduate school project, I'm crafting the Neural Netowrk that I breifly talked about in the title. I've been working on it for a while. This might sound contradictory, but I'm very happy with how I've designed my code... although it does not train the weights correctly. I'm unsure as to why it doesn't. I'm able to customize the amount of nodes in each layer, and amount of layers in the program to be whatever I wish and it will compute an answer. But the answer is always off. I believe the answer to be something that I am structurally not understanding about the Sigmoid activation function, or gradient descent, and fixing the issue won't be something that fundamentally changes how my program is designed algorithmically.
Anyway, I'm asked to have a training set as an input, as well as "correct output", since this is supervised learning (the correct output being very simple, 2*x, where x is the input) that will train the weights around said function. It will not work for the life of me. For example, with an epoch of 50000, ill input data points between 0 and .5, with all of the output values being double that (since the function is 2x, so they will all be between .5 and 1). The final output will be nowhere close to correct, and the weights aren't trained as they should. I believe it to be because the task I've been given might not be entirely possible, I'm interpreting the output data wrong, or the training set I'm using is bad. my advisor wants me to use this activation function to do this task, and I don't want to disappoint.
I believe the answer involves clarification on something that is pretty structural in the understanding of deep learning/sigmoid functions, or what have you. So I didn't go in too much detail when it comes to my design since I don't expect the answer to be a specific issue with my design. However, I'd be more than happy to give more details should someone ask for clarification. Apologies if my post isn't worded too well. Thank you.
