0
$\begingroup$

I'm trying to create an AI using a Neural Network a Genetic Algorithm to learn how to play tetris, but it looks like something is wrong because, even after 20 generations, i can't see any improvement.

The code of the neural network is this:

import numpy

import configuration


def soft_max(z):
    return numpy.exp(z.T) / numpy.sum(numpy.exp(z.T), axis=1).reshape(-1, 1)


def relu(z):
    return numpy.maximum(0, z)


class NeuralNetwork:
    """Class that represent the Neural Network."""

    def __init__(self, weights):
        self.__input_values = numpy.empty(shape=(1, configuration.INPUT))
        self.__weights = weights
        self.__W1_shape = (configuration.NEURONS_HIDDEN_1, configuration.INPUT)
        self.__W2_shape = (configuration.OUTPUT, configuration.NEURONS_HIDDEN_1)
        self.__W1 = self.__weights[0:self.__W1_shape[0] * self.__W1_shape[1]].reshape(self.__W1_shape[0],
                                                                                      self.__W1_shape[1])
        self.__W2 = self.__weights[self.__W1_shape[0] * self.__W1_shape[1]:].reshape(self.__W2_shape[0],
                                                                                     self.__W2_shape[1])

    def update_parameters(self, vision):
        """Update all the input values of the Neural Network."""
        self.__input_values = vision.reshape(-1, configuration.INPUT)

    def forward_propagation(self):
        """Propagates the initial information to the hidden units at each layer and finally produce the output."""
        z1 = numpy.matmul(self.__W1, self.__input_values.T)
        a1 = relu(z1)
        z2 = numpy.matmul(self.__W2, a1)
        a2 = soft_max(z2)
        return a2

    def get_action(self):
        """Based on the output of the forward_propagation produce an Action."""
        return numpy.argmax(numpy.array(self.forward_propagation()))

Where the configuration for those parameters are those one:

# NEURAL NETWORK
INPUT = 234
NEURONS_HIDDEN_1 = 144
OUTPUT = 5
NUMBER_WEIGHTS = INPUT * NEURONS_HIDDEN_1 + NEURONS_HIDDEN_1 * OUTPUT

Input are:

All the board where 0 means block free and 1 means the block is already occupied. The board is 22x10 so those are 220 inputs already. 7 input are for the piece the AI is actually using since you can have 7 different pieces. 4 are for the rotation of the piece 2 are for the X coordinate and Y coordinate.

The output are 5

action = self.get_action_from_nn()
        if action == 0:
            self.shape.move_piece(1)
        if action == 1:
            self.shape.move_piece(-1)
        if action == 2:
            self.shape.drop()
            fast_piece_multiplier = configuration.points['fast_piece_multiplier']
        if action == 3:
            self.shape.rotate()

The last output is "doing nothing"

The genetic algorithm parameters are like this:

# GENETIC ALGORITHM
NUMBER_OF_POPULATION = 1000
NUMBER_OF_GENERATION = 200
NUMBER_PARENTS_CROSSOVER = 50
MUTATION_PERCENTAGE = 0.05

The fitness function is this one:

alfa = -0.510066
beta = 0.760666
charlie = -0.35663
delta = -0.184483

score = alfa * (self.aggregate_height()) + beta * self.total_cleared + charlie * self.holes() + delta * self.bumpiness()
return score

Where self.aggregate_height() calculate the total height of each column inside the board, self.total_cleared are how many rows the AI cleared, self.holes() calculate how many holes are inside the board and self.bumpiness() calculate the difference in height between each pair of columns.

This is not my fitness function, i tried with a lot of different ones and found this one inside a blog where another tetris project was discussed and that was the Fitness function the guy used.

Credit: Tetris Project

$\endgroup$
1
  • $\begingroup$ Any suggestion to obtain better result? $\endgroup$ – Fanto Mar 6 at 9:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.