Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Can neural networks change or evolve other neural networks? Also, could evolutionary algorithms be applied to evolve neural networks?
For example, suppose that we have neural networks A and B. The neural network B changes the neural network A. If B "successfully" changed it, NN A will survive.
Yes, this is an active area of research as we speak. Both using classic algorithms (decision trees, random forests, Bayesian ensembles) as well as neural networks. This can also be done via evolutionary algorithms. I have personally used them for hyperparameter tuning in a few cases where squeezing out a couple of extra points of accuracy was key.
This is in fact what Google is doing with their AutoML system. They are using neural networks for architecture search.
Here is a Github repo with some interesting papers and links on the topic you are describing: https://github.com/hibayesian/awesome-automl-papers.
This answer points at some of the more modern approaches. This has been around for a long time in the form of NeAT: Neuroevolution of Augmenting Topologies, originally described in Kenneth Stanley's 2002 paper.
NeAT is available as a package for many languages, including Python, Java, and C++. The algorithm works as a form of genetic programming. A population of networks is generated with simple, random, topologies. Then they are evaluated according to a loss function for a specific task. The poorly performing networks are discarded, and the better performing ones are intermixed to generate new variations. This process is iterated until the user wishes it to stop, and typically results in gradual improvement of average population performance against the loss function.
Neil Slater is correct when saying that NEAT itself is not neural networks evolving neural networks, what I believe is the closest framework to what the question is asking would be HyperNEAT http://axon.cs.byu.edu/~dan/778/papers/NeuroEvolution/stanley3**.pdf HyperNEAT operates in a very similar way to what you are describing, from a ten thousand foot view the algorithm is as follows: 1)You lay out nodes for a rnn in a Cartesian space, 2d, 3d, whatever you dimension you wish, this set of coordinates is called the substrate. 2)A cppn is queried by passing in two coordinates at a time as input, which gives the cppn a search space of a hypercube in 2x the dimension the coordinates are in (for substrates in space > 2d this is very large) 3)The output of the cppn is used to encode connection, weights, biases, of the rnn coordinates 4)Then the rnn is evaluated by your fitness function and but the evolution (speciation, reproduction, etc) is ran on the cppn that encoded the rnn. So you evolve a population of cppn "genotypes" that encode rnn or cnn "phenotypes".
The third iteration of NEAT is ES-Hyperneat where all you need to layout in the substrate is the input and output layers (Hyperneat you must layout all hidden nodes of the substrate statically). It uses a subdivision tree to subdivide the search spaces and query the subdivided root coordinates of this tree with the cppn just like hyperneat, checking variance along the way to decide if the new node is in a "high information" topological space, to "evolve" hidden nodes into the substrate (rnn).
