Gday guys,

I'am building a game enviroment (picture) where an agent should position the mouse on the screen (cords upper right corner) and then click to shoot a canonball. If the goal (left) is hit. The agent gets a reward based on the elapsed time between this strike and the last one. If three shots missed. The game is done and the environment will reset.

enter image description here

The env is done so far. But now I wonder what the action space should look like. How can I make the agent choose some x and y coordinates? And how can I combine this whith a "shoot" action?

Thanks in advance for any help

  • $\begingroup$ you can learn the mean $\mu$ of normal distribution for $x$ coordinate position and for $y$ coordinate position. Then the action would be sampled coordinates from the normal distribution. You can also learn standard deviation $\sigma$ or it can be fixed. You don't need to model shoot action, agent can shoot immediately to given coordinates. You would need to use policy gradient/actor critic method for this $\endgroup$
    – Brale
    Mar 10 '20 at 11:07

You could make the actions

  1. move up
  2. move down
  3. move left
  4. move right
  5. shoot

Then you can declare the speed of this movement. If you want to go more in depth you can do

  1. Accelerate up
  2. Accelerate down
  3. Accelerate left
  4. Accelerate right
  5. Shoot

This will give control of the speed to the neural networkm but is harder to train, and you should give the speed of the mouse as an input.

  • $\begingroup$ I also thought about that. But since the score is based on the time between two hits I see some problems. Because the main goal is that the agent gets as fast as possible. But when can can only move x much pixels it will lack precision and speed I think... $\endgroup$
    – Voß
    Mar 10 '20 at 10:43
  • $\begingroup$ I don't think it will lack precision. And if you use the acceleration the speed purely depends on how much you accelerate it on every step, and how many frames you wait to make the next step $\endgroup$ Mar 10 '20 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.