I recently started working on very simple machine learning codes in Python and I came across a big problem: teaching the system to improve on its guesses.
So this is what the code is about: I will have a list of organisms with their features stated in numerical values. I want to write a code that identifies that whether the organism is a cat or a fish or neither based on their characteristics. (For example an organism with a high fur value and 4 legs is more probable to be a cat.)
My idea for the neural network is to have five input nodes(for the five characteristics) and 2 output nodes (one for how cat it is and one for how fish it is). The input nodes are multiplied by a weight value and then all summed together to produce one of the output nodes. This repeats itself for the other output. How much the system got wrong is just the difference between the value of the output nodes and how cat/fish the being actually is.
But how can I use this information to correct the weights of the input nodes? Since the weights are randomly generated they can be in the "wrong" directions to begin with. For example if the subject is a cat then we should be expecting high fur and leg values. But what if the weight for the leg value is negative while the weight for the fur value is positive? Adding or multiplying the weight by the error won't bring us any closer to accurately determining the being. Is my neural network flawed to begin with? Or is there a rule of thumb in choosing back propagation algorithms?