11
$\begingroup$

I'm interested in knowing whether there exist any neural network, that solves (with >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.

Improve this question
$\endgroup$
4
  • $\begingroup$ Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer. $\endgroup$
    – nbro
    Mar 1, 2019 at 15:06
  • 2
    $\begingroup$ I think it will not be easy to answer your question. What do you mean by "non-trivial problem"? $\endgroup$
    – nbro
    Mar 1, 2019 at 15:08
  • $\begingroup$ I want few nodes even in hidden layers. $\endgroup$
    – Guillermo Mosse
    Mar 1, 2019 at 20:11
  • $\begingroup$ @nbro my sense of "non-trivial" in this context is intractable or unsolved. $\endgroup$
    – DukeZhou
    Mar 1, 2019 at 22:54

1 Answer 1

Reset to default
13
$\begingroup$

Even if it’s impossible to answer this question properly, as non trivial is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me

Smallest Neural Network to Learn the Ising Criticality

Assuming you have a general idea of the Ising Model I think the problem of identifying the critical temperature from a data driven perspective can be considered as non trivial and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper Machine learning phases of matter from 2017, to only 2 Hidden Neurons

Just my cents:

  • reducing the neurons, while keeping good performance, should help in terms of neural processing interpretability which is notoriously obscure and its complexity grows (exponentially) with the number of neurons
Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right? $\endgroup$
    – Guillermo Mosse
    Mar 1, 2019 at 20:12
  • 1
    $\begingroup$ @GuillermoMosse No, it's quite trivial to define non-trivial. The definition is more or less arbitrary depending on what best fits the situation. I'm sure a very simple NN could learn to tell whether or not a given number is a non-trivial square root... $\endgroup$
    – forest
    Mar 2, 2019 at 3:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .