Is there an efficient way to implement a random crossover of individuals stored in a matrix?

Question

I am using a GA to optimise an ANN in Matlab. This ANN is pretty basic (input, hidden, output) but the input size is quite large (10,000) and the output size is 2 since I have to classes of images to be classified.

The weights are in the form of 2 matrices (10,000*m) and (m * 2). I am now trying to do the genetic cross over with mutation.

Since the weights are in a matrix, is there an efficient way to implement a random crossover (with mutation) without doing it in a point-wise fashion?

Answer 1

Firstly, before we commence I will recommend that you refer to a similar questions on the network https://stackoverflow.com/questions/828486/neural-net-optimize-w-genetic-algorithm

The majority of ML studies focus on gradient algorithms, usually a variation of back-propagation to obtain the weights of the model.

However since genetic algorithms are powerful searching algorithms they can be utilized to tune and optimize the structure and parameters of neural networks.

Genetic algorithms are stochastic search techniques that guide a population of solutions towards an optimum using the principles of evolution and natural genetics.

Genetic algorithms are especially capable of handling problems in which the objective function is discontinuous/non differentiable, nonconvex, multimodal or noisy. (Dharmistha D)

Algorithms, which combine genetic algorithms and error back-propagation, have been shown to exhibit better convergence properties than pure back-propagation.

To facilitate the crossover operation for the exchange of information between two parents we need to define four components within a genetic algorithm.

• A way of coding solutions to the problem on chromosomes;
• A fitness function which returns a value for each chromosome given to it;
• A way of initializing the population of chromosomes;
• Operators that may be applied to parents to alter their genetic composition i.e. mutation.

In these hybrid systems weights in different layers of the network are optimized using a genetic algorithm. Experimental results demonstrate that the genetic algorithms can in some cases optimize the hyper-parameters of an ANN better than a hand tuned model.

Regarding an efficient way to implement a random crossover, here is something to get you started (Code courtesy of Dan Golding 2013)

%Randomly choose 2 individuals

n = size(new_pop, 1);
l = size(new_pop, 2);

breeders = new_pop(randperm(n,2),:);

%Choose a crossover point

cp = randperm(l, 1);

%Crossover

b1 = [breeders(1, 1:cp), breeders(2, cp+1:end)];
b2 = [breeders(2, 1:cp), breeders(1, cp+1:end)];

For further reference I recommend that you kindly look at the below links

Matlab crossover genetic algorithm

https://stackoverflow.com/questions/17696323/matlab-crossover-genetic-algorithm

Single point ordered crossover in matlab

https://stackoverflow.com/questions/16302382/single-point-ordered-crossover-in-matlab

To optimize a neural network of multiple inputs using a genetic algorithm.

https://www.mathworks.com/matlabcentral/answers/180513-to-optimize-a-neural-network-of-multiple-inputs-using-a-genetic-algorithm?s_tid=gn_loc_drop

How can I use the Genetic Algorithm (GA) to train a Neural Network in Neural Network Toolbox

https://www.mathworks.com/matlabcentral/answers/100323-how-can-i-use-the-genetic-algorithm-ga-to-train-a-neural-network-in-neural-network-toolbox