Neural networks consist of so many parameters. Researchers could create as many possible neural networks as they wish. So I want to ask a general question. Could we devise an evolutionary algorithm which learns an efficient structure without optimization?
Are there some important works in this area? If we look at sparse neural networks, it seems that there are so many topologies that perform as well as a dense network.
So a single task has so many solutions which differ slightly. So getting rid of optimization for many problems shouldn't be hard at all.
Edit: I add some more information. I want to know whether we could find sparse topologies by mutating them like adding layers and changing the connections without optimizing the loss function directly?