Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.
12 votes
Accepted

How should I encode the structure of a neural network into a genome?

Section 4.2 of "Essentials of Metaheuristics" has a wealth of information on alternative ways of encoding graph structures via Genetic Algorithms. With particular regard to evolving ANNs, I would ...
user avatar
11 votes
Accepted

How can I automate the choice of the architecture of a neural network for an arbitrary problem?

I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like NEAT (NeuroEvolution of Augmenting Topologies). The ...
user avatar
  • 226
7 votes
Accepted

What if the more fit parent has fewer nodes compared to the other, will the disjoint and excess genes be discarded?

When crossover happens and one parent is fitter than the other, the nodes from the more fit parent are carried over to the child. This is the case as disjoint and excess genes are only carried over ...
user avatar
  • 173
6 votes

How is neural architecture search performed?

You could say that NAS fits into the domain of Meta Learning or Meta Machine learning. I've pulled the NAS papers from my notes, this is a collection of papers/lectures that I personally found very ...
user avatar
5 votes
Accepted

Can neural networks evolve other neural networks?

Yes, this is an active area of research as we speak. Both using classic algorithms (decision trees, random forests, Bayesian ensembles) as well as neural networks. This can also be done via ...
user avatar
5 votes

How does mating take place in NEAT?

NEAT has a constant number of organisms in its population, which prevents overpopulation from happening. The process of mating includes the following steps. The worst networks from every species ...
user avatar
  • 151
5 votes

How can I automate the choice of the architecture of a neural network for an arbitrary problem?

The other answer mentions NEAT to generate network weights or topologies. The paper NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks, which also gives a short ...
user avatar
5 votes

Iteratively and adaptively increasing the network size during training

Neuroevolution Through Augmenting Topologies or NEAT may be what you are referring to. The original paper by Kenneth O. Stanley is here NEAT combines a neural network and a genetic algorithm. Instead ...
user avatar
  • 319
4 votes

How should I encode the structure of a neural network into a genome?

Using evolutionary algorithms to evolve neural networks is called neuroevolution. Some neuroevolution algorithms optimize only the weights of a neural network with fixed topology. That sounds not ...
user avatar
4 votes

Does NEAT require only connection genes to be marked with a global innovation number?

It is actually the other way around: connection IDs is what is debated! Nodes always have innovation IDs (in the image, it is just their identifying number). Node IDs are sufficient to identify ...
user avatar
  • 206
4 votes
Accepted

Is there a neural network with a varying number of neurons?

Yes, NEAT (NeuroEvolution of Augmenting Topologies) increases the number of neurons during training. More specifically, NEAT uses evolution to introduce new neurons and connections during training, ...
user avatar
  • 236
4 votes
Accepted

Can neuroevolution be combined with gradient descent?

The paper The Comparison and Combination of Genetic and Gradient Descent Learning in Recurrent Neural Networks: An Application to Speech Phoneme Classification (2007), by Rohitash Chandra and ...
user avatar
  • 33.2k
4 votes
Accepted

Why evolutionary training of neural networks is not popular?

The main evolutionary algorithm used to train neural networks is Neuro-Evolution of Augmenting Topoloigies, or NEAT. NEAT has seen fairly widespread use. There are thousands of academic papers ...
user avatar
3 votes

Can neural networks evolve other neural networks?

This answer points at some of the more modern approaches. This has been around for a long time in the form of NeAT: Neuroevolution of Augmenting Topologies, originally described in Kenneth Stanley's ...
user avatar
3 votes
Accepted

Several questions regarding the NEAT algorithm

What is the definition of "structural innovation", and how do I store these so I can check if an innovation has already happened before? Structural innovation is anything added that changes the ...
user avatar
3 votes

Does training happen during NEAT?

NEAT uses genetic algorithms both to search for improved connection weights and for improved architectures. Whilst it is possible to train a NEAT-generated neural network using backpropagation of ...
user avatar
  • 23.3k
3 votes

In novelty search, are the novel structures or behaviour of the neural network rewarded?

In the paper Exploiting Open-Endedness to Solve Problems Through the Search for Novelty (2008), by Joel Lehman and Kenneth O. Stanley, which introduced the novelty search approach, it is written Thus ...
user avatar
  • 33.2k
3 votes
Accepted

Can mutation enable a disabled connection?

Yes, the original gene is disabled, but is left in the genome. This can be seen on page 10, figure 3 of the paper linked (taken from the original paper NEAT Paper) where gene 3 is disabled, but not ...
user avatar
3 votes

How to represent the weights of a neural network as binary strings for a genetic algorithm?

I would first say consider the advice of Thomas W in the comment above and think about whether you really need to discretize your variables. I'd also question the wisdom of training a reasonably sized ...
user avatar
  • 446
3 votes

Is it possible to classify data using a genetic algorithm?

It is possible, but is a pretty terrible idea. There are a few options. One is to not use the GA as a direct classifier, but instead use a GA to learn the parameters of another classification model ...
user avatar
  • 446
3 votes

When do mutations in NEAT occur?

When does the mutation occur and how does it take place? Finding a solution in NEAT algorithm is based on evolution strategy. It means that you have Neural Networks which are yours individuals, so ...
user avatar
3 votes
Accepted

Is it possible to perform neuroevolution without a fitness function?

You do not always need an explictly coded fitness function to perform genetic algorithm searches. The more general need is for a selection process that favours individuals that perform better at the ...
user avatar
  • 23.3k
2 votes

What happens if 2 genes have the same connection but a different innovation number?

When do mutations occur and between which nodes? There are two types of mutations in the NEAT model, each of them appears randomly during one epoch on different individuals; the number of structures ...
user avatar
  • 322
2 votes
Accepted

Does neuroevolution require a labelled dataset?

The GA will require a fitness function, which means you need labeled data for comparison. That conclusion is wrong. Yes, sometimes your fitness function will use labeled data. For example, if you ...
user avatar
2 votes

Is it possible to classify data using a genetic algorithm?

You must understand that a genetic algorithm is an optimization algorithm. You can't feed it e-mails and make it classify spam. A genetic algorithm is used to train a model to classify spam. That ...
user avatar
2 votes

How to compute the output of a neural network produced by NEAT?

The networks in NEAT are still implicitly layered. There are neurons that need to be evaluated before other neurons can be evaluated and so this gives us our layers. If you don't know the structure ...
user avatar
2 votes
Accepted

Why would someone use NEAT over other machine learning algorithms?

The main difference leading to strengths and weaknesses of NEAT algorithm, is that it does not use any gradient calculations. That means for NEAT, neither the cost function, nor the activation ...
user avatar
  • 23.3k
2 votes

What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?

$$1-\sum_i(e_i-a_i)^2$$ $\sum$ - there just means sum. It is the greek letter for S. You can rewrite the above formula as $$1 -[(e_1 - a_1)^2+(e_2-a_2)^2+(e_3-a_3)^2+\ldots ]$$ $\sum$ just helps us ...
user avatar
2 votes

Can neural networks evolve other neural networks?

Neil Slater is correct when saying that NEAT itself is not neural networks evolving neural networks, what I believe is the closest framework to what the question is asking would be HyperNEAT http://...
user avatar
  • 317
2 votes
Accepted

How can non-functional neural networks be avoided when the crossover produces a child with a disabled gene?

In your example, the output node would still get a value from Input1, even though Input2 is disabled. If the child was: ...
user avatar
  • 319

Only top scored, non community-wiki answers of a minimum length are eligible