How to make a bot to learn from previous games

Question

How to train a bot, given a series of games in which he did (initially random) actions, to improve its behavior based on previous experiences?

The bot has some actions: e.g. shoot, wait, move, etc. It's a turn based "game" in which, for know, I'm running the bots with some objectives (e.g. kill some other bot) and random actions. So every bot will have a score function that at the end of the game will say, from X to Y (0 to 100?) if they did well or not.

So how to make the bots to learn of their previous experiences? Because this is not a fixed input as the neural networks take, this is kind of a list of games, each one in which the bot took several actions (one by every "turn"). The IA functions that I know are used to predict future values.. I'm not sure is the same.

Maybe I should have a function that gets the "more similar previous games" that the bot played and checked what were the actions he took, if the results were bad he should take another action, if the results were good then he should take the same action. But this seems kind of hardcoded.

Another option would be to train a neural network (somehow fixing the problem of the fixed input) based on previous game actions and to predict the future action's results in score (something that I guess it's similar to how chess and Go games work) and choose the one that seems to have better outcome.

I hope this is not too abstract. I don't want to hardcode much stuff in the bots, I'd like them to learn by their own starting from a blank page.

Answer 1

Reinforcement learning

The problem that you describe, namely, choosing a good sequence of actions based on a reward/score received based on the whole sequence (and possibly significantly delayed), is pretty much the textbook definition of Reinforcement learning.

As with quite a few other topics, deep neural networks currently seem to be a promising way for solving this type of problems. This may be a beginner-friendly description of this approach.

Answer 2

If it is a game you can try a simple weightage calculation where if the bot perform an action that yields a positive result - killed an enemy, gained an advantageous position etc. Add a 'weight' to that action that in similar circumstances the chances of performing that action that will lead to a positive result is higher.

Yet due to the chance of not performing an action that was remembered to yield positive results, there is a little bit of 'randomness' and also a chance to discover new possibilities. Just remember not to let a single occurrence shift the weightage too much or allow a single action's weightage to become so high that the AI stops trying different actions on similar situations.