0
$\begingroup$

Neural networks consist of so many parameters. Researchers could create as many possible neural networks as they wish. So I want to ask a general question. Could we devise an evolutionary algorithm which learns an efficient structure without optimization?

Are there some important works in this area? If we look at sparse neural networks, it seems that there are so many topologies that perform as well as a dense network.

So a single task has so many solutions which differ slightly. So getting rid of optimization for many problems shouldn't be hard at all.

Edit: I add some more information. I want to know whether we could find sparse topologies by mutating them like adding layers and changing the connections without optimizing the loss function directly?

Improve this question
$\endgroup$
4
  • $\begingroup$ I was thinking about this subject after reading this paper: ceur-ws.org/Vol-2993/paper-10.pdf. The paper suggests that it might be possible to create generative neural networks which don't need backpropagation or any optimization. If it could be proven that we only need to devise an evolutionary algorithm for many tasks, we no longer need resource-heavy optimization. $\endgroup$
    – bitWise
    Dec 25, 2021 at 7:40
  • 1
    $\begingroup$ Are you aware of hyper-parameter optimization techniques? Moreover, what do you mean by "without optimization"? Evolutionary algorithms are also optimization algorithms, but you're proposing to use EAs to solve this problem, but then you say "without optimiztion", so it's not clear what you have in mind. Could you please edit your post to clarify what you're really looking for? $\endgroup$
    – nbro
    Dec 25, 2021 at 9:12
  • 1
    $\begingroup$ Please, put your specific question in the title. Your question is not the one in the title "Is it possible to find a good neural network structure without training it?". Your question seems to be "could we find sparse topologies by mutating them like adding layers and changing the connections without optimizing the loss function directly?". Having said that, it seems that your question is answered here and here. Let me know if that's the case. If not, edit your post to clarify why not. $\endgroup$
    – nbro
    Dec 25, 2021 at 10:02
  • $\begingroup$ Don't forget to upvote those posts if you think they are useful! Meanwhile, I closed this post. Feel free to ask another question, if you were looking for something else, but please be clearer and specific. $\endgroup$
    – nbro
    Dec 25, 2021 at 10:05

0

Reset to default

Browse other questions tagged .