I am making a very simple neural network for a school project, and I would like to know what the best and easiest way to "teach" a neural network would be. From what I know, backpropagation requires a lot of mathematical knowledge. Whilst Generational is gene-based and creating mutations doesn't. If there is another way to do it, that would be great to hear. I just want someone else's opinion and perspective before I start programming.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Training neural network by backpropagation doesn't require much mathematics, especially with the plethora of libraries available. With the libraries available, it would be more computationally efficient and tuning parameters would be a lot easier. This heavily contrast with Genetic Algorithms where not a lot of libraries are available and not enjoying as much learning resources. One thing about Genetic algorithm though is that you don't need a differentiable performance function and you don't need human data. But even in those areas, Deep Reinforment Learning algorithms, which still uses backpropagation, far exceeds the performance of evolutionary algorithms.
$\begingroup$
$\endgroup$
From what I know, backpropagation requires a lot of mathematical knowledge.
It's not a huge amount of knowledge. High school (USA) or A-level (UK) maths is enough to understand the principles. There is a lot of detail, and that may take time you don't want to spend just yet. If you use a pre-made library for the neural network though, you don't have to study this in any depth. Instead, you can think of the maths for the learning stage as a library function that you supply correct input/output pairs to, that you run multiple times and monitor results.
In addition, although a genetic algorithm is perhaps a simpler principle to explain and understand, getting the most basic version to work with a neural network task is quite hard in practice, and may take some time to tune.
Provided you are not wanting to build the network logic from scratch, I would recommend using a library either way (for genetic algorithms and neural networks combined for a small network, I would recommend the NEAT algorithm and a library that implements that). The libraries hide a lot of complexity of the maths involved, and there is still plenty to learn and understand about the principles of using machine learning to approximate a function.
Ultimately, what to choose depends on the nature of your project. You don't want to be over-ambitious, but also want to have something interesting in your write-up:
If your primary goal is to write a network from scratch, you should pick a really simple goal for the neural network - a typical toy problem is to learn to copy the XOR function. That will significantly limit the amount of maths you need to study to get it to work from first principles.
If your primary goal is to use a neural network to "do something cool", then you could use library code and tackle a toy problem such as recognising hand-written digits (the MNIST dataset).
One nice ting about using a genetic algorithm is that you can run it on simple simulated games, because you can base the GA improvements based on results from using the network, as opposed to needing to know correct input/output pairs for training. A controller for a simplified game, like flappy bird or for a model car driving around a track is possible in a small project.
