13
$\begingroup$

Assume that I want to solve an issue with a neural network that either I can't fit to already existing topologies (perceptron, Konohen, etc) or I'm simply not aware of the existence of those or I'm unable to understand their mechanics and I rely on my own instead.

How can I automate the choice of the topology (that is, the number of layers, the type of activations, the type and direction of the connections, etc.) of a neural network for an arbitrary problem?

I'm a beginner, yet I realized that in some topologies (or, at least in perceptrons) it is very hard if not impossible to understand the inner mechanics as the neurons of the hidden layers don't express any mathematically meaningful context.

$\endgroup$
11
$\begingroup$

I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like NEAT (NeuroEvolution of Augmenting Topologies).

The original NEAT paper involves evolving weights for connections, but if you only want a topology, you can use a weighting algorithm instead. You can also mix activation functions if you aren't sure which to use. Here is an example of using backpropagation and multiple neuron types.

$\endgroup$
5
$\begingroup$

The other answer mentions NEAT to generate network weights or topologies. The paper NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks, which also gives a short summary of neuroevolution techniques, provides an alternative approach to NEAT. It uses Cartesian Genetic Programming to evolve a multiple activation functions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.