2
$\begingroup$

I have created a game based on this game here. I am attempting to use Deep Q Learning to do this, and this is my first foray into Neural networks (please be gentle!!)

I am trying to create a NN that can play this game. Here are some relevant facts about the game:

  • Player 1 (the fox) has 1 piece that he can move diagonally 1 step in any direction

  • Player 2(The geese) has 4 pieces that they can move only forward diagonally (either diagonal left or diagonal right) 1 step.

  • The Fox wins if he reaches the other end of the board, the geese win if they trap the fox so it cannot move.

I am trying to work on the agent first for the geese as it seems to be the harder agent with more pieces and restrictions. Here is the important sections of code I have so far:

This is where I setup the game board, and set the total actions for the geese

def __init__(self):
    self.state_size = (LENGTH,LENGTH) ##LENGTH is 8 so (8,8)
    #...
    #other DQN variables that aren't important to question
    #...
    self.action_size = 8 ##4 geese, each can potentially make 2 moves
    self.model = self.build_model()

And here is where I create my model

def build_model(self):
    #builds the NN for Deep-Q Model
    model = Sequential() #establishes a feed forward NN
    model.add(Dense(64,input_shape = (LENGTH,), activation='relu'))
    model.add(Dense(64, activation='relu'))
    model.add(Dense(self.action_size, activation = 'linear'))
    model.compile(loss='mse', optimizer='Adam')

This is where I perform an action

def act(self, state,env):
    #get the list of allowed actions for the geese
    actions_allowed = env.allowed_actions_geese_agent()

    if np.random.rand(0,1) <= self.epsilon: ##do a random move
        return actions_allowed[random.randint(0, len(actions_allowed)-1)]
    act_values = self.model.predict(state)
    print(act_values)
    return np.argmax(act_values)

My question: Since there are 4 geese and each can make 2 possible moves, am I correct in thinking that my action_size should be 8 (2 for each goose) or should it be maybe 2 (for diagonal left or right) or something else entirely?

The reason why I am at a loss is because on any given turn, some of the geese may have an invalid move, does that matter?

My next Question: Even if I have the right output layer for the geese agent, when I call model.predict(state) where I pick my action...how do I interpret the output? And how would I map that action it selects to a valid action that can be made?

Here is a picture of the result of using model.predict(state), as you can see it returns a ton of data and then when I call return np.argmax(act_values) I get 59 back...not sure how to utilize that (or if it's even correct based on my output layer)... and finally I included a drawing of the board. F is the fox and 1,2,3,4 are the different geese. enter image description here

I apologize for the massive post, but I am just trying to provide as much information that is helpful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.