I'm trying to understand how to effectively plan and write a Neural Network but running into problems with understanding how they should be written. I'm working with classification with writing a backpropagation Neural Network however any other types/examples would be greatly appreciated.

One of my problems is understanding the possibilities of them, for example if you have a single neuron I assume you can have 2^1 possibilities, like in my example I can't imagine how I could predict something more than above or below a straight line. Similarly I assume with 3 neurons therefore 2^3 possibilities you could predict if something is one of the 4 quadrants of a graph and then predict if it's in the upper part or lower part of the quadrant so 8 classifications. Is this true or is a network capable of predicting more/less than that?

On to my specific problem.

I've written a single Neuron/Perceptron that can predict whether something is above or below a straight line graph given the correct training data and using a sign activation function.

Following from this I'm trying to write a Neural Network that can predict whether something is in the 1st, 2nd, 3rd or 4th quadrant of a graph.

One idea I have had is to have 2 input perceptrons, the first taking the x value, the 2nd taking the y value, these then try and predict individually whether the answer is on the right or left of the centre, and then above or below respectively. These then pass their outputs to the 3rd and final output neuron. The 3rd Neuron uses the inputs to try and predict which quadrant the coordinates are in. The first two inputs use the sign function.

The problems I'm having with this is to do with the activation function of the final neuron, one idea was to have a function that somehow scaled the output into a integer between 0 and 1, so 0 to 0.25 would be quadrant 1, and so on up to 1. Another idea would be to convert it to a value using sin and representing it as a sine wave as this could potentially represent all 4 quadrants.

Another idea would be to have a single neuron taking the input of the x and y value and predicting whether something was above or below a graph (like my perceptron example), then having two output neurons, which the 1st output neuron would be fired if it was above the line and then passed in the original x coordinate to that output neuron. The 2nd output neuron would be fired if it was below then pass in the original x value as well to determine if it was left or right.

Are these ideas adequate examples of writing a network?

Also my final question would be if you wanted your network to have 8 possible outputs would you need 8 output neurons or could you represent the 8 values with a single more advanced activation function?



First of all what you are trying to do can be achieved by simple logical programming. Secondly you are making things overly complex.

1 node in a Neural Net can predict as many values as you would like it to, as it outputs a real number no matter the activation function. It is us who round of the number to 0 or 1 or maybe 0, 0.5, 1 depending on the classification job at hand.

2 Neurons can predict 4 classes 00, 01, 10, 11 depending on how you use it. Now in your specific of quadrant detection 2 Neurons are sufficient, since not only can you decompose the problem into two separate problems, both the problems are linearly separable too. This is not always the case or it maybe too complex to find linearly separable classes, so in general one output node is reserved for 1 class only. If it is that class only that output node is activated while the rest is 0. Memory and process consuming, but easier.

Coming to your first approach. I don't exactly how hope to achieve 4 class classification with a single output node. Minimum of 2 nodes is required. The problem with your approach is that if you think critically your hidden nodes are not doing anything other than scaling the inputs, so almost the same type input is propagated and then you are asking a node to perform a a 4 task classification which is not linearly separable as per me. sin will not be able to separate it if you think critically.

The second idea is better, just remove the hidden node and you will be able to classify it based on 00, 01, 10, 11 outputs. Although this problem can be solved by simple logical computation and your Neural Net is basically doing nothing.

Please point out to me if you find any mathematical mistakes. Hope this helps!

  • $\begingroup$ Yeah the problem is very basic, I'm just trying to predict the quadrants because i'm still trying to understand how to build a network with more than one neuron. Do you any ideas for an easy and useful application of a network I could try? $\endgroup$ – W22 Feb 2 '18 at 13:19
  • $\begingroup$ @W22 if you re using perceptron learning....build your own 2d data-set by randomly initializing nos...don't make it linearly separable...and see what happens..don't use a neural net for now...just a simple one node classifier no hidden network..then plot the decision boundary...see if the decision boundary is visually appealing...if not why? I actually asked a question about this ai.stackexchange.com/questions/4399/… $\endgroup$ – DuttaA Feb 2 '18 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.