I'm trying to understand how to effectively plan and write a Neural Network but running into problems with understanding how they should be written. I'm working with classification with writing a backpropagation Neural Network however any other types/examples would be greatly appreciated.
One of my problems is understanding the possibilities of them, for example if you have a single neuron I assume you can have 2^1 possibilities, like in my example I can't imagine how I could predict something more than above or below a straight line. Similarly I assume with 3 neurons therefore 2^3 possibilities you could predict if something is one of the 4 quadrants of a graph and then predict if it's in the upper part or lower part of the quadrant so 8 classifications. Is this true or is a network capable of predicting more/less than that?
On to my specific problem.
I've written a single Neuron/Perceptron that can predict whether something is above or below a straight line graph given the correct training data and using a sign activation function.
Following from this I'm trying to write a Neural Network that can predict whether something is in the 1st, 2nd, 3rd or 4th quadrant of a graph.
One idea I have had is to have 2 input perceptrons, the first taking the x value, the 2nd taking the y value, these then try and predict individually whether the answer is on the right or left of the centre, and then above or below respectively. These then pass their outputs to the 3rd and final output neuron. The 3rd Neuron uses the inputs to try and predict which quadrant the coordinates are in. The first two inputs use the sign function.
The problems I'm having with this is to do with the activation function of the final neuron, one idea was to have a function that somehow scaled the output into a integer between 0 and 1, so 0 to 0.25 would be quadrant 1, and so on up to 1. Another idea would be to convert it to a value using sin and representing it as a sine wave as this could potentially represent all 4 quadrants.
Another idea would be to have a single neuron taking the input of the x and y value and predicting whether something was above or below a graph (like my perceptron example), then having two output neurons, which the 1st output neuron would be fired if it was above the line and then passed in the original x coordinate to that output neuron. The 2nd output neuron would be fired if it was below then pass in the original x value as well to determine if it was left or right.
Are these ideas adequate examples of writing a network?
Also my final question would be if you wanted your network to have 8 possible outputs would you need 8 output neurons or could you represent the 8 values with a single more advanced activation function?