0
$\begingroup$

A single neuron will be able to do linear separation. For example, XOR simulator network:

x1 --- n1.1
   \  /    \
    \/      \
             n2.1 
    /\      /
   /  \    /
x2 --- n1.2 

Where x1, x2 are the 2 inputs, n1.1 and n1.2 are the 2 neurons in hidden layer, and n2.1 is the output neuron.

The output neuron n2.1 does a linear separation. How about the 2 neurons in hidden layer?

Is it still called linear separation (at 2 nodes and join the 2 separation lines)? or polynomial separation of degree 2?

I'm confused about how it's called because there are curvy lines in this wiki article: https://en.wikipedia.org/wiki/Overfitting

Some curvy separation?

Some other curvy separation?

$\endgroup$
0
$\begingroup$

What you have depicted is a nonlinear classificator. Although each stage does a linear separation, the sequential composition of linear separations is nonlinear. The nonlinearity of the neuron is key in this regard, as otherwise it would all be equivalent to a matrix multiplication, which is linear. You were right about the degree, although it's rarely called like that. People usually describe just the number of layers, and I guess the main reason is that it's not directly equivalent to such polynomials, as that depends on other factors (e.g. the activation function).

$\endgroup$
0
0
$\begingroup$

I found out the curvy zigzag green line is not polynomial as if it were polynomial, a vertical line won't cut that curvy line more than 1 time.

It's the combination of straight lines (linear separation) of multiple neurons in the same layer. So it's linear separation ('linear' by previous_layer_output*weight, 'separation' by activation function), at multiple nodes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.