I understand that a neural network basically distorts(non-linear transformation) and changes the perspective(linear transformations) of input space to draw a plane to classify data. How does the network deduce if an input is one side of a plane and therefore output the decision? Thanks in advance.
3
-
$\begingroup$ Excellent foundation question. Welcome to AI! $\endgroup$– DukeZhouCommented Sep 3, 2017 at 21:46
-
$\begingroup$ @DukeZhou Have an idea? XD $\endgroup$– DanielCommented Sep 4, 2017 at 17:28
-
$\begingroup$ I wish. (All of my work is currently in rules based and axiomatic AI, with reinforcement of heavily bounded rationality still on the horizon.) $\endgroup$– DukeZhouCommented Sep 5, 2017 at 20:58
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
If you are working with supervised learning, each training example has a label. That label is your classification of the provided input. Just like linear or logistic regression, if your problem only has 2 classes (e.g. determine whether a tumor is malignant or not), your network will have a single output. An output value of 1.0 could represent one class and an output of 0.0 represents the other. This is basically determining which side of multi-dimensional plane your input is on.
If you have more than 2 (m) classes, then you will have m outputs from your network. Each output will represent the likelihood of the input matching that class. For example if you have an image recognition network that has 4 classes: Dog, truck, boat, person, you would have 4 outputs with each one representing how likely that image is one of the classes. Think about this as 4 independent planes, where each one is making a determination: does this image match this class(1 of {dog, truck, boat, person}) or not. If you had outputs of dog: 0.3, truck: 0.25, boat: 0.7, person: 0.4, One simple algorithm would be simply pick the class with the maximum value -- in this case, it would classify this image as a boat.
-
$\begingroup$ Thanks for the answer. After the plane for classification is drawn, what part part of the algorithm determines which side of the place the point lies? $\endgroup$– DanielCommented Sep 8, 2017 at 18:43
-
$\begingroup$ The output could be a single number (binary classification) or a vector (multiple classification) with values from 0 to 1. A threshold comes in place to the first, a threshold is a parameter number [0,1], and you can choice based on the consequences you prefer (see ROC curve), or in the case of the vector you can assign the class of the maximum number. As said Paul, you could pass the output vector for a function, let’s call
assign_max_class(output_vector)$\endgroup$ Commented Nov 8, 2017 at 0:30
$\begingroup$
$\endgroup$
4
For other who were wondering the same questions as me, I'll answer it.
My view above was inconsistent. Ultimately the last layer of simple feed-forward networks don't have any special properties previous layers exhibit. NNs are just glorified mathematical functions. It distorts space with linear(matrix multiplies) and non-linear functions.
Theres no 'decision plane' per-say, only function mappings up to the very end where we want to map the input to (in this case binary classification problem) to two separate numbers (usually 1 or 0).
Hope this clear things up for people getting into NNs.
-
$\begingroup$ Is your inference that output layer does not contain any special properties correct? $\endgroup$– user9947Commented Feb 3, 2018 at 6:54
-
$\begingroup$ @DukeZhou I think Neural Nets without any hidden layer can perform some basic classification tasks..am i wrong? $\endgroup$– user9947Commented Feb 3, 2018 at 6:55
-
$\begingroup$ Yes it is correct. It works on the same basis as any other layer except it it usually uses a special non-linear function that 'sorts' the input into classification categories. $\endgroup$– DanielCommented Feb 7, 2018 at 22:40
-
$\begingroup$ Also, you're right on your assumption, a NN without any hidden layers can preform classification. This network is actually the simple logistic classification algorithm. $\endgroup$– DanielCommented Feb 7, 2018 at 22:42
$\begingroup$
$\endgroup$
So I think in case of a logistic regression task a Neural Network works something like this.
First of all I think all nodes perform the job of mapping a point to a quadrant, in a n-space co-ordinate system where the n-space is decided iteratively by the problem statement itself. In short the nodes decide which combination of polynomial terms of the input matter in the task at hand. In a hidden layer since we don't perform a classification task it outputs real number. But the output layer rounds off numbers as per the classification task.
If you have a single hidden layer its nodes can be thought of outputting values that matter. It gives the NN a degree of freedom. Like in Machine Learning we select the polynomial combinations, a NN selects it by itself iteratively. But if see the function of output nodes, it just rounds of the number to 0 or 1. Thus output nodes can perform simple tasks of classification (like which quadrant a point lies in) without a hidden layer. I believe if we can decipher what the hidden nodes want to convey us we can entirely remove the output nodes. Because hidden nodes convey information like, in which quadrant a point lies only in a cryptic format which is resolved by the output nodes iteratively.
After knowing the information we can easily resolve it by logical statements.
But this odes not demean the power of output nodes. In case of Linear Regression a single output node without any hidden layer can approximate a quarter of the sine wave (by adjusting the exponent of e to make t look like a sine wave). So it only depends on how you use the Neural Network.
But the basic principle is the same a node decides whether an input is positive or negative (in case of logistic activation) or outputs a real number depending on how you use it.
If you find anything contradictory please correct me.