I have created a game based on this game here. I am attempting to use Deep Q Learning to do this, and this is my first foray into Neural networks (please be gentle!!)
I am trying to create a NN that can play this game. Here are some relevant facts about the game:
Player 1 (the fox) has 1 piece that he can move diagonally 1 step in any direction
Player 2(The geese) has 4 pieces that they can move only forward diagonally (either diagonal left or diagonal right) 1 step.
The Fox wins if he reaches the other end of the board, the geese win if they trap the fox so it cannot move.
I am trying to work on the agent first for the geese as it seems to be the harder agent with more pieces and restrictions. Here is the important sections of code I have so far:
This is where I setup the game board, and set the total actions for the geese
def __init__(self): self.state_size = (LENGTH,LENGTH) ##LENGTH is 8 so (8,8) #... #other DQN variables that aren't important to question #... self.action_size = 8 ##4 geese, each can potentially make 2 moves self.model = self.build_model()
And here is where I create my model
def build_model(self): #builds the NN for Deep-Q Model model = Sequential() #establishes a feed forward NN model.add(Dense(64,input_shape = (LENGTH,), activation='relu')) model.add(Dense(64, activation='relu')) model.add(Dense(self.action_size, activation = 'linear')) model.compile(loss='mse', optimizer='Adam')
This is where I perform an action
def act(self, state,env): #get the list of allowed actions for the geese actions_allowed = env.allowed_actions_geese_agent() if np.random.rand(0,1) <= self.epsilon: ##do a random move return actions_allowed[random.randint(0, len(actions_allowed)-1)] act_values = self.model.predict(state) print(act_values) return np.argmax(act_values)
My question: Since there are 4 geese and each can make 2 possible moves, am I correct in thinking that my action_size should be 8 (2 for each goose) or should it be maybe 2 (for diagonal left or right) or something else entirely?
The reason why I am at a loss is because on any given turn, some of the geese may have an invalid move, does that matter?
My next Question: Even if I have the right output layer for the geese agent, when I call
model.predict(state)where I pick my action...how do I interpret the output? And how would I map that action it selects to a valid action that can be made?
Here is a picture of the result of using
model.predict(state), as you can see it returns a ton of data and then when I call
return np.argmax(act_values) I get 59 back...not sure how to utilize that (or if it's even correct based on my output layer)... and finally I included a drawing of the board. F is the fox and 1,2,3,4 are the different geese.
I apologize for the massive post, but I am just trying to provide as much information that is helpful.