0
$\begingroup$

A single neuron will be able to do linear separation. For example, XOR simulator network:

x1 --- n1.1
   \  /    \
    \/      \
             n2.1 
    /\      /
   /  \    /
x2 --- n1.2 

Where x1, x2 are the 2 inputs, n1.1 and n1.2 are the 2 neurons in hidden layer, and n2.1 is the output neuron.

The output neuron n2.1 does a linear separation. How about the 2 neurons in hidden layer?

Is it still called linear separation (at 2 nodes and join the 2 separation lines)? or polynomial separation of degree 2?

I'm confused about how it's called because there are curvy lines in this wiki article: https://en.wikipedia.org/wiki/Overfitting

Some curvy separation?

Some other curvy separation?

$\endgroup$

2 Answers 2

0
$\begingroup$

What you have depicted is a nonlinear classificator. Although each stage does a linear separation, the sequential composition of linear separations is nonlinear. The nonlinearity of the neuron is key in this regard, as otherwise it would all be equivalent to a matrix multiplication, which is linear. You were right about the degree, although it's rarely called like that. People usually describe just the number of layers, and I guess the main reason is that it's not directly equivalent to such polynomials, as that depends on other factors (e.g. the activation function).

$\endgroup$
0
0
$\begingroup$

I found out the curvy zigzag green line is not polynomial as if it were polynomial, a vertical line won't cut that curvy line more than 1 time.

It's the combination of straight lines (linear separation) of multiple neurons in the same layer. So it's linear separation ('linear' by previous_layer_output*weight, 'separation' by activation function), at multiple nodes.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .