Are there neural networks that can decide to add/delete neurons (or change the neuron models/activation functions or change the assigned meaning for neurons), links or even complete layers during execution time? I guess, that such neural networks overcome the usual separation of learning/inference phases and they continuously live their lives in which learning and self-improving occurs alongside performing inference and actual decision making for which these neural networks were build. Effectively it could be neural network that acts as a [Gödel machine] (

I have found the term dynamic neural network but it is connected to adding some delay functions and nothing more.

Of course, such self-improving networks completely redefine the learning strategy, possibly, single shot gradient methods can not be applicable to them.

My question is connected to the neural-symbolic integration, e.g. Neural-Symbolic Cognitive Reasoning by Artur S. D'Avila Garcez, 2009. Usually this approach assigns individual neurons to the variables (or groups of neurons to the formula/rule) in the set of formulas in some knowledge base. Of course, if knowledge base expands (e.g. from sensor readings or from inner nonmonotonic inference) then new variables should be added and hence the neural network should be expanded (or contracted) as well.

  • 1
    as far as my experience takes me, you are looking for genetic algorithms, they keep a population of neural networks and merge them together at random points given how good they do – oren revenge Sep 14 at 10:36
  • But such external GAs are external controllers which are incapable of learning. I am most interested in self-aware neural networks that themselves can take those decisions. – TomR Sep 14 at 10:39
  • 1
    maybe look at this : – Jérémy Blain Sep 14 at 12:39

Your Answer


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.