2
$\begingroup$

I'm trying to make a bot to the famous "Icy Tower" game. I rebuilt the game using pygame and I'm trying to build the bot using Python-NEAT.

Every generation a population of 70 characters tries to jump to the next platform and increase their fitness. right now the fitness is the number of platforms they jumped on, each platform gives +10.

example

The problem I'm facing is that the bot isn't learning good enough after 1000 generations the best score was around 200 (it can get to 200 even at the first few generations by mistake. 200 means 20 platforms which is not a lot).

when I look at the characters jumping it looks like they just always jump and go left or right and not deliberately aiming to the next platform.

I tried several input configurations to make the bot perform better. but nothing really helped.

these are the inputs I tried to mess around with:

  • pos.x, pos.y
  • velocity.x, velocity.y
  • isOnPlatform (bool)
  • [plat.x, plat.y, plat.width] (list of the 3-7 next platforms locations, also tried distance from character in x,y)
  • [prev.x, prev.y] (2-6 previous character positions)

I'm not so proficient with neuroevolution and I'm probably doing something wrong. glad if you could explain what's causing the bot to be so bad or what's not helping him to learn properly.

Although I think that the fitness function and the inputs should be the only problem I'm attaching the python-NEAT config file.

[NEAT]
fitness_criterion     = max
fitness_threshold     = 10000
pop_size              = 70
reset_on_extinction   = False

[DefaultGenome]
# node activation options
activation_default      = tanh
activation_mutate_rate  = 0.0
activation_options      = tanh

# node aggregation options
aggregation_default     = sum
aggregation_mutate_rate = 0.0
aggregation_options     = sum

# node bias options
bias_init_mean          = 0.0
bias_init_stdev         = 1.0
bias_max_value          = 30.0
bias_min_value          = -30.0
bias_mutate_power       = 0.5
bias_mutate_rate        = 0.7
bias_replace_rate       = 0.1

# genome compatibility options
compatibility_disjoint_coefficient = 1.0
compatibility_weight_coefficient   = 0.5

# connection add/remove rates
conn_add_prob           = 0.5
conn_delete_prob        = 0.5

# connection enable options
enabled_default         = True
enabled_mutate_rate     = 0.01

feed_forward            = True
initial_connection      = full

# node add/remove rates
node_add_prob           = 0.2
node_delete_prob        = 0.2

# network parameters
num_hidden              = 6
num_inputs              = 11
num_outputs             = 3

# node response options
response_init_mean      = 1.0
response_init_stdev     = 0.0
response_max_value      = 30.0
response_min_value      = -30.0
response_mutate_power   = 0.0
response_mutate_rate    = 0.0
response_replace_rate   = 0.0

# connection weight options
weight_init_mean        = 0.0
weight_init_stdev       = 1.0
weight_max_value        = 30
weight_min_value        = -30
weight_mutate_power     = 0.5
weight_mutate_rate      = 0.8
weight_replace_rate     = 0.1

[DefaultSpeciesSet]
compatibility_threshold = 3.0

[DefaultStagnation]
species_fitness_func = max
max_stagnation       = 3
species_elitism      = 2

[DefaultReproduction]
elitism            = 3
survival_threshold = 0.2
  • Note: the previous character positions are the position in the previous frame, and if the game runs at 60 fps the previous position is not that different from the current one...

  • Note2: the game score is a bit more complex than just jumping on platforms, the bot should also be rewarded for combos that can make him jump higher. the combo system is already implemented but I first want to see the bot aiming to the next platform before he learns to jump combo.

$\endgroup$
  • $\begingroup$ Is platforms' arrengment all the time the same or it changes every time, or maybe every couple of times? $\endgroup$ – Makintosz May 15 at 8:03
  • $\begingroup$ The platform location is random every time. but even if it wasn't, the bot can't memorize the level's layout. right? $\endgroup$ – Lidor shimoni May 15 at 20:08
  • $\begingroup$ You should try evolution with all the time constant arrangement of the platform. I don't know what are your parameters that you try to find, but if this is the list of movements that your character do in the game (step by step) then of course it will not work, because the fitness function of one of such movement list will be totally irrelevant on a different platform configuration. What means that the results from one generation will not be the same in the next one, therefore evolution can not take place. $\endgroup$ – Makintosz May 16 at 9:38
  • $\begingroup$ @Makintosz I'm not quite sure I understand you, I saw bots for "flappy bird" with random levels and it worked with NEAT. why it won't work on my game? why training on the same level layout should work? the bot cant memorizes as far as I know. $\endgroup$ – Lidor shimoni May 16 at 11:55
  • $\begingroup$ I think it depends on how you set it up, and what is that you feed to the network. Situation you described seems like randomness is the key here. What do you feed the netwrok with? $\endgroup$ – Makintosz May 16 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.