2
$\begingroup$

I am developing a NEAT flappy bird game, and it doesn't work, the system stays stupid for 300 generations. I chose tanh() for activation, just because it's included in JS.

I can't find a good discussion on the internet of activation functions in the context of neuroevolution, most of what I see is about derivative and other gradient descent issues which I suspect are irrelevant to forward only networks.

If you need a fixed point to answer, I have 8 inputs, one output and the problem is a classification ("jump", "don't jump"). But please explain your answer. I currently use tanh() for all the hidden and output nodes, and the output is considered "jump" if the output neuron value is >0.85

For some context, the code is here: https://github.com/nraynaud/nraygame and the game here: https://nraynaud.github.io/nraygame/

| improve this question | |
$\endgroup$
  • 1
    $\begingroup$ Why do you use only 1 output neuron? Why not use 2 output neurons? 1 for jump and 1 for not jump? $\endgroup$ – Lustwelpintje Feb 24 at 12:24
  • $\begingroup$ it feels more binary to have number and a threshold, and simpler to have less nodes, but if you have a good explanation as to why 2 is better than one, I just want to learn. $\endgroup$ – nraynaud Feb 24 at 12:39
  • 2
    $\begingroup$ I don't have many experience in NEAT, but I have made some reinforcement learning NNs. When you are training your NN using the MDP logic,Your output is the value of each action. If you have 2 output neurons this would be the value of jumping and the value of not jumping. when comparing these 2 values, you can then take the action with the biggest number. When you only have one output neuron, you can't train these values.However, when you use supervised learning instead of reinforcement learning, you can use your method (because the system learns from your predefined data, not from experience). $\endgroup$ – Lustwelpintje Feb 24 at 13:49
  • 1
    $\begingroup$ I just checked, NEAT works with reinforcement learning methods. So using 2 output neurons could make it learn better. $\endgroup$ – Lustwelpintje Feb 24 at 13:52
  • 1
    $\begingroup$ Reinforcement learning works by training the network with the values of the actions. If you have 1 output neuron you will train that output neuron to be higher if you got far and lower if you died immediatly. In this case, it will be trained randomly. If you have 2 output neurons, you will do a run, select a action, and train 1 of those output neurons (which will be the action you selected). This way, the other output neuron will be unaffected. In your way, the network won't learn the 2 preceding hidden neurons the way you want to, because it is trained on the reward on the next output neuron. $\endgroup$ – Lustwelpintje Feb 25 at 10:24
0
$\begingroup$

When it comes to genetic algorithms (neuroevolution), you can pretty much evolve any kind of parameters. You just need to have a fairly complex system where each parameter changes the way the inputs are affecting the outputs.

So, to answer your question, any activation function should work! It would be surprising that the activation function is your problem. But if you want to test some activation functions, here's a list of activation functions. But tanh should work perfectly fine.

As I already mentioned in another answer, the key to have a great neuroevolution algorithm is the fitness function. Your rewards should reflect the goal you're aiming for. You didn't mentioned what fitness function you used, but here's what I suggest: reward a player ONLY if he progresses in the game. For example, your fitness function could simply be the distance traveled by your birds. This could maybe solve your problem...

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.