-2
$\begingroup$

If the number of input neurons and output neurons doesn't change, what will change if I have one hidden layer, but first with 1 neuron, then with 4 neurons?

Taking into consideration the fact that each perceptron is able to linearly separate points on an unknown/unwritten linear function, would this then be able to, theoretically, instead of simply linearly separate points, separate points into those that occur inside a square, and those that occur outside?

This is, of course, without a bias neuron present.

$\endgroup$
  • $\begingroup$ StackExchange works way better if you ask only a single question per Post. Also, please focus on the question. Sometimes context is important; in this case very likely not. $\endgroup$ – Martin Thoma Nov 15 '18 at 6:45
  • $\begingroup$ Thank you, Martin Thoma. I will take the advice to heart. $\endgroup$ – Gareth Nov 16 '18 at 4:59
  • $\begingroup$ Can those who down vote,tell us why? $\endgroup$ – quintumnia Nov 16 '18 at 16:15
0
$\begingroup$

If the number of input neurons and output neurons doesn't change, what will change if I have one hidden layer, but first with 1 neuron, then with 4 neurons?

I recommend that you try to implement a dense layer with Tensorflow (not using the Dense implementation, of course).

A dense layer is the full connection between to layers. What actually is learned/optimized in a neural network are the weights in between layers. Each weight is just a single floating point number. It's the connection between two nodes. If you increase the number of nodes of one layer, the two neighboring layers get more weights. Hence the optimization algorithm can adapt better.

For simplicity, I'll now assume you don't use biases.

If you have A input neurons and Z output neurons, a single hidden layer with 1 neutron means you have two matrices of shape Ax1 and 1xZ. if you change it to 4 hidden neurons the two matrices have the shape Ax4 and 4xZ

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.