I want to train a DQN card game named witches. It consists of 60 Cards (1-14 of Yellow, Blue, Green, Red Cards + 4 Wizzards). The color of the first layed card has to be respected by the other players (if they have this card in hand). The one who has the card with the highest number gets the played cards. Each collected red card gives you a -1 point.
With respect to this answer I setup the inputs / state of the NN as binary (meaning I have 180 bool values (60 card x is currently on the table, 60 card x is in the ai hand, 60 card x is already been played)
How to design the outputs / actions?
- If the ai is the first player of a round it can play any card
- If the ai is not first player it has to respect the first played card (or play a wizzard)
This means there is actually a list of available options. I then sort this list and have 60 Output bools which I set to 1 if this option is possible. Among these options the ai should then decide what the correct option is? Is this the correct procedure?
Inconsistent / Varying Action Space This is what we have to deal here with. As explained in here I think a DQN as well as Policy Gradient Methods is not the correct architecture to choose for solving such multi-agent card games. What architecture would you choose?
Assume I have 4 players, so do I have to get the old state before the ai is the next player and the new state is directly after this round is finished?
my_game = game(["Tim", "Bob", "Lena", "Anja"]) while True: #1. Play unti AI has to move: my_game.play_round_until_ai() #2. Get old state: state_old = agent.get_state(my_game) #3. Get the action the AI should perform action_ = agent.get_action(state_old, my_game) #4. perform new Action and get new state #reward rates how good the current action was #score is the actual score of this game! reward, done, score = my_game.finishRound(action_) # 5: Calculate new state state_new = agent.get_state(my_game) #6. train short memory base on the new action and state agent.train_short_memory(state_old, action_, reward, state_new, done) #7. store the new data into a long term memory agent.remember(state_old, action_, reward, state_new, done) if done == True: # One game is over, train on the memory and plot the result. sc = my_game.reset()
My code so far is available here: https://github.com/CesMak/witches_ai