What are examples of approaches to create an AI for a fighting robot in an MMO game?

Question

I have an MMO game where I have players. I wanted to invent something new to the game, and add player-bots to make the game be single-playable as well. The AI I want to add is simply only for fighting other players or other player-bots that he sees around at his level.

So, I thought of implementing my fighting strategy, exactly how I play, to the bot which is basically using if statements and randoms. For example, when the opponent has low health, and the bot has enough special attack power, he will use this chance and use his special attack power in order to try to knock the opponent down, or if the bot has low health he will eat in time but not too much because there is a point in risking fights, if you eat too much your opponent will do too. Or, for example, if the bot detects the opponent player is eating too much and gains health, he will do the same.

I told this idea of the implementation to one of my friends and he simply responded with: This is not AI, it's simply just a set of conditions, it does not have any heuristic functions.

For that type of game, what are some ideas to create a real AI to achieve these conditions?

Basically, the AI should know what to do in order to beat the opponent, based on the opponent's data such as current health, Armour and weapons, and level, if he risks his health or not and so on.

I am a beginner and it really interests me to do it in the right way.

Answer 1

I would set up a list of goals for your bot. These could be 'maintain a minimum level of health', 'knock out human player', 'block way to location X', etc. This obviously depends on the domain of your MMO.

Then you can use a planner to achieve these goals in the game. You define a set of actions with preconditions and effects, set the current goal, and the planner will work out a list of actions for the bot to achieve the goal. You can easily express your actions (and the domain) in PDDL.

Examples for actions would be 'move to location X', 'eat X', 'attack player X'. A precondition of 'attack player X' could be 'health(X) is low', and an effect could be 'health(X) is reduced by Y'. There are different ways of expressing these depending on the planner's capabilities.

The beauty of this is that you don't actually have to explicitly code any behaviour. You describe the domain, and tell the bot what it should achieve, and what capabilities it has. The actual behaviour then emerges out of that description. If the bot only attacks a player if the player has lower health, then observing the player eat (and thus up their health) could result in the bot eating (to push its own health above the player's so that it can attack) — but you have not told the bot directly to do that.

For a starting point, go to http://education.planning.domains/ for a list of resources.

If you only have a few actions available, it might appear predictable to a human user, but with a variety of goals and actions, this will quickly become more complex and seem more 'intelligent'.

Update: Here is a link to a paper, Applying Goal-Oriented Action Planning to Games, which describes how this can be applied in a game.

Answer 2

Oliver Mason's answer is great for specific methods and tools to use, but I wanted to pull out a more general principle which was mentioned in a comment.

The distinction your friend is making is not one that would be generally recognised. One of my university lecturers defined AI as something like "an artificial system that exhibits behaviour that resembles how an intelligent being would behave".

If an intelligent being would always use the special attack in a particular situation, then an algorithm that always does so in the same situation is behaving intelligently, even though the algorithm behind it is incredibly simple. If you can come up with a complete description of an intelligent player, you have what is called an expert system, i.e. a system which captures the decision-making process of a real expert.

Your friend is not even correct that your proposed AI "does not have any heuristic functions". When you write a condition like "if the AI's health is below 50%, it will eat food", you're approximating the rule a human would use. You can make the heuristic more complex by increasing the probability of eating in proportion to current health; that might in turn make the heuristic closer to optimal.

You can only really find out how "good" your AI is by putting it into different situations and observing it - sometimes, a simple set of rules gives rise to "emergent behaviours" that look surprisingly intelligent. As you build up more complex rules - i.e. more optimal heuristics - the emergent behaviour will change, and you can tweak it for the desired effect.

Answer 3

You can train your bot using reinforcement learning (in particular Q-Learning).

The most important part of the RL is a reward function. If we want agent to do some thing specific, we must provide rewards to it in such a way that it will achieve our goals. It is thus very important that the reward function accurately indicates the exact behaviour

So you can construct your own reward function that will satisfy your requirements. If the bot does something you want, you will reward it with higher score, otherwise you will punish it with negative reward.

AlhpaGo and OpenAI teams used a similar technique to train their model which could then beat humans in games like Go, StarCraft 2 and Dota 2

Also, check out this Deep Reinforcement Learning Free Preview on udacity.

Answer 4

If you are willing to take an evolutionary approach, you may employ the NEAT algorithm (Neural Evolution of Augmenting Topologies) to train your bot. It will take some work setting it up and all, but it then will gradually improve over time.

Check out the following:

• http://gekkoquant.com/2016/03/13/evolving-neural-networks-through-augmenting-topologies-part-1-of-4/
• https://www.youtube.com/watch?v=WiPZSieT6qs
• https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies

That should be enough to pique your interests and get you started. That last link links to a number of NEAT implementations available in a number of languages.

Answer 5

What about GANs or genetic algorithms?

The first idea (GAN) is that you basically create 2+ random bots who fight each other, and they keep adjusting their weights so that they can beat the other bot. That means, that those 2+ bots keep improving their "fighting performance" for as long as you want, eventually being even better than humans.

The second idea (genetic algorithms) is to generate a lot of bots who genetically differ from each other just for a slight mutation. You make them fight, and the best perforing/last standing will become the new clone where the new "lot of bots" gets generated from.