The neural network for a board game with somewhat imperfect information

Question

I am a Software Engineer and I implemented online a pretty complex board game. Some key rules of the game.

1. You have some cards in the hand and you can choose different turns.
2. Build a card of certain type, which would increase your income or give victory points.
3. You can choose to receive the resources equal to your income.
4. You can choose to take more cards.
5. Some cards that you build have pretty complex effects that can't be measured, some examples are: increase the number of cards you can take during a single turn, a card that lets you make a special turn if you have that card etc.

What I already tried to do. I tried to create a state of the game that would include

• income of every player
• current number of resources of every player
• current victory points
• the state of the game that affects how long it might take till the end
• the number of current turn, since it is variable and depends on the play

I would collect this data for every game, then mark which data belongs to a player who won, and who lost and teach the model.

Then I took this model, calculated every possible turn and future state if a player made that turn and then I would evaluate which of those turn would show the highest chance of a win. And then I would consider that turn the most optimal.

The problem with this approach is that it doesn't evaluate those special cards with special effects, because they simply don't fit into that model.

Also the most complex part is that player may choose which cards he wants to take and which cards he wants to discard. How can I teach the engine to evaluate the value of that card? For example a certain card may be extremely good but there might be some limitations according to player current situation. I don't know how to put this data into the model.

Also I ignore the fact that opposite player can play something, even though I can evaluate his current game situation, because I can see everything he played, or has, except his hand.

In the end I simply made my computer play vs itself and collected data on which cards played on which turn show the best win rate for a player in the end.

But I want to make and teach a model, maybe not by evaluating the board, because I can't fit those complex cards, but maybe some other solution. Give me please any advice at which approaches/literate to look.

Answer 1

You probably want to have a look at reinforcement learning. I will not be able to cover this complex topic in one post, but I will give you some key ideas (and yes, this is a simplification and experts of the field might have a valid point that there are exceptions to what I am writing):

1. You are already half-way there. Reinforcement learning aims to predict the gain of each possible action.
2. What - from my understanding of your description - might be missing is the fact, that reinforcement learning how to estimate the gain of each action. This is done by evaluating the outcome of each game. You can image an iterative process:
   Actions / States that lead to winning the game will be rates positive, those that lead to losing the game are negative. If this is learned, then actions that lead to a positive state will be rated positive and others will be rated negative.
   By doing so, the system can learn the long-term effect of actions even though they have now direct gain. In your case, it might learn that having these special cards is a better state than not having them (and also when it is a good action to play them)
3. Reinforcement learning can deal with uncertainties (e.g. the unknown cards of the other players, what kind of action they might take). A well designed reinforcement learning system optimizes for the best expectation.
4. For the training, the system has to play against other computer players. The do not necessarily have to use the same strategy.

For further insides and practical applications I would suggest to read some tutorials, introductions or books about the topic. As I said, the topic is complex.