5
$\begingroup$

If I have a set of sensory nodes taking in information and a set of "action nodes" which determine the behavior of my robot, why do I need hidden nodes between them when I can let all sensory nodes affect all action nodes?

(This is in the context of evolving neural network)

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Look for the idea called "Hierarchical representation" $\endgroup$
    – Ankur
    Commented Oct 24, 2016 at 6:55

3 Answers 3

Reset to default
5
$\begingroup$

A feed forward neural network without hidden nodes can only find linear decision boundaries. However, most of the time you need non-linear decision boundaries. Hence you need hidden nodes with a non-linear activation function. The more hidden nodes you have, the more data you need to find good parameters, but the more complex decision boundaries you can find.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ It appears that the topic is much more complex than I initially thought. What field of study is this in, and is there a name for this specific aspect of neural networking? $\endgroup$
    – user289661
    Commented Nov 1, 2016 at 3:44
  • 1
    $\begingroup$ @user289661 This is one of the most basic aspects of artificial neural networks. It's in machine learning. $\endgroup$
    – Martin Thoma
    Commented Nov 1, 2016 at 10:41
  • 1
    $\begingroup$ And you generally want more than one hidden layer, because while a sufficiently large network can capture any function to an arbitrary precision, for certain computations a shallow network would need exponentially more nodes than a one layer deeper network that does the same. $\endgroup$
    – Peteris
    Commented Nov 1, 2016 at 21:12
  • 1
    $\begingroup$ @Peteris While I agree that you generally want more than one hidden layer, I wonder if you have a source for the claim that a shallow network needs exponentially more nodes than a one layer deeper network. $\endgroup$
    – Martin Thoma
    Commented Nov 1, 2016 at 22:05
  • 1
    $\begingroup$ @MartinThoma arxiv.org/pdf/1603.00988.pdf or arxiv.org/pdf/1509.08101.pdf are some sources; the particular proofs of course are different for different types of functions; It generally starts with Hastad 1987 "Computational limitations of small-depth circuits" and in general this concept is the counterpoint to the universal approximator theorem - while yes, even the simplest architectures can implement anything, for certain functions they require an unreasonable amount of resources. $\endgroup$
    – Peteris
    Commented Nov 1, 2016 at 23:17
1
$\begingroup$

Neural Networks are very good approaches for robots. The main function of Neural Net is to model the interdependence between all the features. Now this can be done manually by selecting possible combinations of features between themselves upto a certain degree. But this approach has drawbacks:

  • It is tedious to go about selecting features.
  • It costs time and additional computer resources to calculate the values of the new features you have introduced.
  • Since you cannot visualize data more than 3-D you cannot be absolutely sure that your selected features are enough to model your problem.

Now if you use an NN, the NN will automatically select the combination of features (provided it has enough hidden nodes) by adjusting the weights of connections between and the features and nodes. The main advantages of this approach are:

  • You don't have to manually select the feature combinations.
  • If data is still not fitting you can easily increase or decrease the number of nodes without needing to modify the whole network.
  • Also it will be computationally efficient since you don't have to calculate values of factors that don't matter to the problem.

Hope this is what you were looking for!

Improve this answer
$\endgroup$
1
$\begingroup$

Normally one node/layer applies linear fitting of the the input to the hypothesis, in other words uses linear function ($y = ax + b$). Adding layers chains liner functions, potentially allowing fitting higher order functions. A great explanation can be found here.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .