I'm making an artificial intelligence for a card game using MCTS. With a standard 52-cards deck, 4 hands are dealt: 1 for each of the 3 players and one extra hand. Then, each player gets the choice to trade its hand with the extra hand or keep its current one. No one is allowed to see the extra hand before trading.

I initially tried to run MCTS on the initial game state, but because of UCT, one of the two choices (to trade or not to trade) was being quickly eliminated due to random initial odds. For example, after 1000 simulations, I had something like 990 visits on the trade node and 10 on the don't trade node. Tweaking the exploration parameter didn't help much.

So instead I only ran the simulation step in a way that the two choices were tested equally, eg: 500 times each. It worked great.

The problem is, there's an obvious penalty to trading your hand after someone else traded already, because this person now knows your hand. But since the simulation step only makes random moves, only the potential value of hand is taken into account.

Is there any way to tweak the MCTS simulation step so that it takes that penalty into account? The penalty comes from the fact that AI players only randomize the state of the game that they don't know, but each card played and each hand seen in left as is.

I do realize that MCTS may not be suitable for this situation. If so, what would be the best option?

I want to avoid introducing a heuristic like dampening the simulation score depending on how many players have traded their hand before. (eg: after simulation, multiply the average score by 50% for the "trade" move if one player has already traded.)

Note that there's usually a human player in the game and that my goal is not to make a superhuman AI, just to improve my current solution a little.

  • 1
    $\begingroup$ I think you should include in the state space that a card has been seen by all players (i.e. was traded). This should allow the learning system to distinguish between the cases were cards were seen by others. $\endgroup$ – agold Aug 27 '19 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.