I have created a game based on this game here. I am attempting to use Deep Q Learning to do this, and this is my first foray into Neural networks (please be gentle!!)

I am trying to create a NN that can play this game. Here are some relevant facts about the game:

  • Player 1 (the fox) has 1 piece that he can move diagonally 1 step in any direction

  • Player 2(The geese) has 4 pieces that they can move only forward diagonally (either diagonal left or diagonal right) 1 step.

  • The Fox wins if he reaches the other end of the board, the geese win if they trap the fox so it cannot move.

I am trying to work on the agent first for the geese as it seems to be the harder agent with more pieces and restrictions. Here is the important sections of code I have so far:

This is where I setup the game board, and set the total actions for the geese

def __init__(self):
    self.state_size = (LENGTH,LENGTH) ##LENGTH is 8 so (8,8)
    #other DQN variables that aren't important to question
    self.action_size = 8 ##4 geese, each can potentially make 2 moves
    self.model = self.build_model()

And here is where I create my model

def build_model(self):
    #builds the NN for Deep-Q Model
    model = Sequential() #establishes a feed forward NN
    model.add(Dense(64,input_shape = (LENGTH,), activation='relu'))
    model.add(Dense(64, activation='relu'))
    model.add(Dense(self.action_size, activation = 'linear'))
    model.compile(loss='mse', optimizer='Adam')

This is where I perform an action

def act(self, state,env):
    #get the list of allowed actions for the geese
    actions_allowed = env.allowed_actions_geese_agent()

    if np.random.rand(0,1) <= self.epsilon: ##do a random move
        return actions_allowed[random.randint(0, len(actions_allowed)-1)]
    act_values = self.model.predict(state)
    return np.argmax(act_values)

My question: Since there are 4 geese and each can make 2 possible moves, am I correct in thinking that my action_size should be 8 (2 for each goose) or should it be maybe 2 (for diagonal left or right) or something else entirely?

The reason why I am at a loss is because on any given turn, some of the geese may have an invalid move, does that matter?

My next Question: Even if I have the right output layer for the geese agent, when I call model.predict(state) where I pick my action...how do I interpret the output? And how would I map that action it selects to a valid action that can be made?

Here is a picture of the result of using model.predict(state), as you can see it returns a ton of data and then when I call return np.argmax(act_values) I get 59 back...not sure how to utilize that (or if it's even correct based on my output layer)... and finally I included a drawing of the board. F is the fox and 1,2,3,4 are the different geese. enter image description here

I apologize for the massive post, but I am just trying to provide as much information that is helpful.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.