I'm trying to create an AI using a Neural Network a Genetic Algorithm to learn how to play tetris, but it looks like something is wrong because, even after 20 generations, i can't see any improvement.
The code of the neural network is this:
import numpy import configuration def soft_max(z): return numpy.exp(z.T) / numpy.sum(numpy.exp(z.T), axis=1).reshape(-1, 1) def relu(z): return numpy.maximum(0, z) class NeuralNetwork: """Class that represent the Neural Network.""" def __init__(self, weights): self.__input_values = numpy.empty(shape=(1, configuration.INPUT)) self.__weights = weights self.__W1_shape = (configuration.NEURONS_HIDDEN_1, configuration.INPUT) self.__W2_shape = (configuration.OUTPUT, configuration.NEURONS_HIDDEN_1) self.__W1 = self.__weights[0:self.__W1_shape * self.__W1_shape].reshape(self.__W1_shape, self.__W1_shape) self.__W2 = self.__weights[self.__W1_shape * self.__W1_shape:].reshape(self.__W2_shape, self.__W2_shape) def update_parameters(self, vision): """Update all the input values of the Neural Network.""" self.__input_values = vision.reshape(-1, configuration.INPUT) def forward_propagation(self): """Propagates the initial information to the hidden units at each layer and finally produce the output.""" z1 = numpy.matmul(self.__W1, self.__input_values.T) a1 = relu(z1) z2 = numpy.matmul(self.__W2, a1) a2 = soft_max(z2) return a2 def get_action(self): """Based on the output of the forward_propagation produce an Action.""" return numpy.argmax(numpy.array(self.forward_propagation()))
Where the configuration for those parameters are those one:
# NEURAL NETWORK INPUT = 234 NEURONS_HIDDEN_1 = 144 OUTPUT = 5 NUMBER_WEIGHTS = INPUT * NEURONS_HIDDEN_1 + NEURONS_HIDDEN_1 * OUTPUT
All the board where 0 means block free and 1 means the block is already occupied. The board is 22x10 so those are 220 inputs already. 7 input are for the piece the AI is actually using since you can have 7 different pieces. 4 are for the rotation of the piece 2 are for the X coordinate and Y coordinate.
The output are 5
action = self.get_action_from_nn() if action == 0: self.shape.move_piece(1) if action == 1: self.shape.move_piece(-1) if action == 2: self.shape.drop() fast_piece_multiplier = configuration.points['fast_piece_multiplier'] if action == 3: self.shape.rotate()
The last output is "doing nothing"
The genetic algorithm parameters are like this:
# GENETIC ALGORITHM NUMBER_OF_POPULATION = 1000 NUMBER_OF_GENERATION = 200 NUMBER_PARENTS_CROSSOVER = 50 MUTATION_PERCENTAGE = 0.05
The fitness function is this one:
alfa = -0.510066 beta = 0.760666 charlie = -0.35663 delta = -0.184483 score = alfa * (self.aggregate_height()) + beta * self.total_cleared + charlie * self.holes() + delta * self.bumpiness() return score
self.aggregate_height() calculate the total height of each column inside the board,
self.total_cleared are how many rows the AI cleared,
self.holes() calculate how many holes are inside the board and
self.bumpiness() calculate the difference in height between each pair of columns.
This is not my fitness function, i tried with a lot of different ones and found this one inside a blog where another tetris project was discussed and that was the Fitness function the guy used.
Credit: Tetris Project