How to use Machine Learning with simple games?

Question

I built a simple HTML game. In this game the goal is to click when the blue ball is above the red ball. If you hit, you get 1 point, if you miss, you lose 1 point. With each hit, the blue ball moves faster. You can test the game here.

Without using machine learning, I would easily solve this problem by just clicking when the X, Y of the blue ball was on the X, Y of the red ball. Regardless of the time, knowing the positions of the 2 elements I could solve the problem of the game.

However, if I wanted to create an AI to solve this problem, could I? How would it be? I'd really like to see the AI randomly wandering until it's perfect.

My way to solve the problem

I click many times and watch score. If score down, add to bad_positions. If actual position in bad_positions, not click. At first he misses many times, then starts to hit eternally. This is machine learning? Deep learning? Just a bot?

var bad_positions = [];
function train(){
  var pos = $ball.offset().left;
  var last_score = score;
  if (!bad_positions.includes(pos)) {
   $('#hit').click();
    if (score < last_score){
      bad_positions.push(pos)
    } 
  }
}

Answer 1

You have implemented a simple contextual bandit solver, which is a machine learning algorithm. A few details may be different from a full implementation, but the key elements are:

• A choice of actions (click hit or don't click hit)

• A reward signal that can be observed after each action (+1 for a hit, 0 for nothing happens, -1 for an attack which misses)

• An observable state which affects the reward achievable (the position of the blue ball). For a contextual bandit, the state is not influenced by the action taken. This is true here.

• One thing that is different about your problem from a classic contextual bandit is that the next state is predictable from the current state (whilst in a pure bandit problem it should be entirely random). However, that's not too important to your problem here, and your solver is definitely following a contextual bandit approach.

• Your solver tests the score from trying different actions in each state, and narrows down the best action to take in each state. Your implementation is simple and "greedy" for a contextual bandit solver. A more typical solution would maintain an average result for each action and have a rule for how to explore actions in each state, so it could test whether results were reliable (this is very helpful with non-deterministic scenarios where bandit solvers are more often used).

With each hit, the blue ball moves faster

Unless you somehow limit the reaction time of the agent, this is not relevant to how you write the solver. You could change the rules affecting the agent to make it relevant in the same way as it would be for a human, e.g. deciding to click means the click happens 0.1 seconds later, and the state can include observations of position just now and several 0.02 seconds going back.

In general, if you want to take this further, with more complex games and still learning how to control agent actions, you could look at simple reinforcement learning agents, such as Q-learning. If you are interested in the underlying theory of agents like this, then a good (and free) introductory text is Sutton & Barto "Reinforcement Learning: An Introduction"