A single neuron will be able to do linear separation. For example, XOR simulator network:
x1 --- n1.1 \ / \ \/ \ n2.1 /\ / / \ / x2 --- n1.2
x2 are the 2 inputs,
n1.2 are the 2 neurons in hidden layer, and
n2.1 is the output neuron.
The output neuron
n2.1 does a linear separation. How about the 2 neurons in hidden layer?
Is it still called linear separation (at 2 nodes and join the 2 separation lines)? or polynomial separation of degree 2?
I'm confused about how it's called because there are curvy lines in this wiki article: https://en.wikipedia.org/wiki/Overfitting