So, i have created Snake game using Pygame and Python. Then i wanted to create an AI with Genetic algorithm and a simple NN to play it. Seems pretty fun, but things aren't working out.
This is my genetic algorithm:
def calculate_fitness(population): """Calculate the fitness value for the entire population of the generation.""" # First we create all_fit, an empty array, at the start. Then we proceed to start the chromosome x and we will # calculate his fit_value. Then we will insert, inside the all_fit array, all the fit_values for each chromosome # of the population and return the array all_fit =  for i in range(len(population)): fit_value = Fitness().fitness(population[i]) all_fit.append(fit_value) return all_fit def select_best_individuals(population, fitness): """Select X number of best parents based on their fitness score.""" # Create an empty array of the size of number_parents_crossover and the shape of the weights # after that we need to create an array with x number of the best parents, where x is NUMBER_PARENTS_CROSSOVER # inside config file. Then we search for the fittest parents inside the fitness array created by the # calculate_fitness function. Numpy.where return (array(, dtype=int64),) that satisfy the query, so we # take only the first element of the array and then it's value (the index inside fitness array). After we have # the index of the element we just need to take all the weights of that chromosome and insert them as a new # parent. Finally we change the fitness value of the fitness value of that chromosome inside the fitness # array in order to have all different parents and not only the fittest parents = numpy.empty((config.NUMBER_PARENTS_CROSSOVER, population.shape)) for parent_num in range(config.NUMBER_PARENTS_CROSSOVER): index_fittest = numpy.where(fitness == numpy.max(fitness)) index_fittest = index_fittest parents[parent_num, :] = population[index_fittest, :] fitness[index_fittest] = -99999 return parents def crossover(parents, offspring_size): """Create a crossover of the best parents.""" # First we start by creating and empty array with the size equal to offspring_size we want. The type of the # array is [ [Index, Weights] ]. If the parents size is only 1 than we can't make crossover and we return # the parent itself, otherwise we select 2 random parents and then mix their weights based on a probability offspring = numpy.empty(offspring_size) if parents.shape == 1: offspring = parents else: for offspring_index in range(offspring_size): while True: index_parent_1 = random.randint(0, parents.shape - 1) index_parent_2 = random.randint(0, parents.shape - 1) if index_parent_1 != index_parent_2: for weight_index in range(offspring_size): if random.uniform(0, 1) < 0.5: offspring[offspring_index, weight_index] = parents[index_parent_1, weight_index] else: offspring[offspring_index, weight_index] = parents[index_parent_2, weight_index] break return offspring def mutation(offspring_crossover): """Mutating the offsprings generated from crossover to maintain variation in the population.""" # We cycle though the offspring_crossover population and we change x random weights, where x is a parameter # inside the config file. We select a random index, generate a random value between -1 and 1 and then # we sum the original weight with the random_value, so that we have a variation inside the population for offspring_index in range(offspring_crossover.shape): for _ in range(offspring_crossover.shape): if random.uniform(0, 1) == config.MUTATION_PERCENTAGE: index = random.randint(0, offspring_crossover.shape - 1) random_value = numpy.random.choice(numpy.arange(-1, 1, step=0.001), size=1, replace=False) offspring_crossover[offspring_index, index] = offspring_crossover[offspring_index, index] + random_value return offspring_crossover
My neural network is formed using 7 inputs:
is_left_blocked, is_front_blocked, is_right_blocked, apple_direction_vector_normalized_x, snake_direction_vector_normalized_x, apple_direction_vector_normalized_y,snake_direction_vector_normalized_y
Basically if you can go left, front, right, direction to the apple and snake direction. Then i have an hidden layer with 8 neurons and finally 3 output that indicate left, keep going or right.
The Neural Network forward() is calculate like this:
self.get_weights_from_encoded() Z1 = numpy.matmul(self.__W1, self.__input_values.T) A1 = numpy.tanh(Z1) Z2 = numpy.matmul(self.__W2, A1) A2 = self.sigmoid(Z2) A2 = self.softmax(A2) return A2
where self.__W1 and self.__W2 are the weights from input to hidden layer and then the weights from hidden layer to the output. Softmax(A2) return the index of the matrix[1,3] where the value is the biggest, then i use that index to indicate the direction that my neural network choose.
This is the config file that contains the parameters:
# GENETIC ALGORITHM NUMBER_OF_POPULATION = 500 NUMBER_OF_GENERATION = 200 NUMBER_PARENTS_CROSSOVER = 50 MUTATION_PERCENTAGE = 0.2 # NEURAL NETWORK INPUT = 7 NEURONS_HIDDEN_1 = 8 OUTPUT = 3 NUMBER_WEIGHTS = INPUT * NEURONS_HIDDEN_1 + NEURONS_HIDDEN_1 * OUTPUT
And this is the main:
for generation in range(config.NUMBER_OF_GENERATION): snakes_fitness = genetic_algorithm.calculate_fitness(population) # Selecting the best parents in the population. parents = genetic_algorithm.select_best_individuals(population, snakes_fitness) # Generating next generation using crossover. offspring_crossover = genetic_algorithm.crossover(parents, offspring_size=(pop_size - parents.shape, config.NUMBER_WEIGHTS)) # Adding some variations to the offspring using mutation. offspring_mutation = genetic_algorithm.mutation(offspring_crossover) # Creating the new population based on the parents and offspring. population[0:parents.shape, :] = parents population[parents.shape:, :] = offspring_mutation
I have 2 problems:
1) I don't see an improvement over the new generations
2) I'm actually running the game inside the for loop, but waiting for all the snake of a generation to die and repeat with the new one is really time consuming. Isn't there a way to launch all or, atleast, more than 1 instance of the game and keep filling the array with the result?
This is Fitness().fitness(population[i])
def fitness(self, weights): game_manager = GameManager(weights) self.__score = game_manager.play_game() return self.__score
This is where it's called inside the for loop
def calculate_fitness(population): """Calculate the fitness value for the entire population of the generation.""" # First we create all_fit, an empty array, at the start. Then we proceed to start the chromosome x and we will # calculate his fit_value. Then we will insert, inside the all_fit array, all the fit_values for each chromosome # of the population and return the array all_fit =  for i in range(len(population)): fit_value = Fitness().fitness(population[i]) all_fit.append(fit_value) return all_fit
This the function that launch the game (GameManager(weights)) and return the score of the snake.
This is my first time on AI so this code could be all a mess, don't worry about pointing out what i did wrong, just please don't say "It's all wrong" because i won't be able to learn otherwise.