I'm learning about multilayer perceptrons, and I have a quick theory question in regards to hidden layer neurons.
I know we can use two hidden layers to solve a non-linearly seperable problem by allowing for a representation with two linear seperators. However, we can solve a non-linearly seperable problem using only one hidden layer.
This seems fine, but what kind of representation does one hidden layer add? My question is how is the dimensionality of the output affected?
I've drawn a diagram of a multilayer perceptron with one hidden layer neuron. I used this same layout to solve a non-linearly seperable problem. The single hidden layer node is inside the red square. Forgive my poor MS-Paint skills.