I failed using PPO to train a multiplayer card game. Thus I tested monte carlo tree search (mcts) to predict good moves. This works now (you can test the game here. As calculating a good move using mcts takes quite long, I wondered if I could use the predicted good moves from mcts and the input state to train a neuronal network.
I would use the following to train the neuronal network:
- Input State [played cards, cards on table, cards in the hand of the player] 60 Cards in game thus this is a 180x1 binary vector
- Output State [60x1 vector of the best predicted move from mcts, 4x1 rewards vector of the 4 players]
What I basically would like to achieve is a Neuronal Network that would predict me the action I should play and the estimated output rewards at the end of the game for a given Input State.
Whats Data should I generate to efficiently train the network?
- Should I just use the good moves mcts is predicting or should I use also the bad ones?
- Should I setup the game players to be all mcts / or random players?
- Is it possible that the Network does learn anything at all from this data?