# Mapping Actions to the Output Layer in Keras Model for a Board Game

I have created a game based on this game here. I am attempting to use Deep Q Learning to do this, and this is my first foray into Neural networks (please be gentle!!)

I am trying to create a NN that can play this game. Here are some relevant facts about the game:

• Player 1 (the fox) has 1 piece that he can move diagonally 1 step in any direction

• Player 2(The geese) has 4 pieces that they can move only forward diagonally (either diagonal left or diagonal right) 1 step.

• The Fox wins if he reaches the other end of the board, the geese win if they trap the fox so it cannot move.

I am trying to work on the agent first for the geese as it seems to be the harder agent with more pieces and restrictions. Here is the important sections of code I have so far:

This is where I setup the game board, and set the total actions for the geese

def __init__(self):
self.state_size = (LENGTH,LENGTH) ##LENGTH is 8 so (8,8)
#...
#other DQN variables that aren't important to question
#...
self.action_size = 8 ##4 geese, each can potentially make 2 moves
self.model = self.build_model()


And here is where I create my model

def build_model(self):
#builds the NN for Deep-Q Model
model = Sequential() #establishes a feed forward NN


This is where I perform an action

def act(self, state,env):
#get the list of allowed actions for the geese
actions_allowed = env.allowed_actions_geese_agent()

if np.random.rand(0,1) <= self.epsilon: ##do a random move
return actions_allowed[random.randint(0, len(actions_allowed)-1)]
act_values = self.model.predict(state)
print(act_values)
return np.argmax(act_values)


My question: Since there are 4 geese and each can make 2 possible moves, am I correct in thinking that my action_size should be 8 (2 for each goose) or should it be maybe 2 (for diagonal left or right) or something else entirely?

The reason why I am at a loss is because on any given turn, some of the geese may have an invalid move, does that matter?

My next Question: Even if I have the right output layer for the geese agent, when I call model.predict(state) where I pick my action...how do I interpret the output? And how would I map that action it selects to a valid action that can be made?

Here is a picture of the result of using model.predict(state), as you can see it returns a ton of data and then when I call return np.argmax(act_values) I get 59 back...not sure how to utilize that (or if it's even correct based on my output layer)... and finally I included a drawing of the board. F is the fox and 1,2,3,4 are the different geese.

I apologize for the massive post, but I am just trying to provide as much information that is helpful.