11
$\begingroup$

I am currently trying to train a bot for a game I am creating. It is a 2d game with a complex map made of various shapes. The bot and character shoot bullets that are capable of ricocheting. The neural network outputs a vector in which the bot will turn and then fire. I myself cannot calculate the correct trajectory and find the Loss of the network. But, I can give a rating on how well the neural network performs when it fires. Can I train it by simply giving it a rating, and if, how so?

$\endgroup$
2
  • $\begingroup$ @foreverska, can I just rate it on the damage the bot deals to the character? Higher damage = lower loss and vice-versa, then just back-propagate? $\endgroup$
    – Beluker
    Commented Apr 8 at 2:26
  • 3
    $\begingroup$ No, it's more complex than that, although there are simplified reinforcement learning approaches. If your action choices are discrete, you might like to look into the Cross Entropy Method (CEM), which is where you run e.g. 100 episodes, take the top 10 performing ones and treat every action taken in them as being correct to train a multi-class classifier. Clearly that's not true, but the small degree to which it is true (some of the actions will be better choices) can be enough to train the neural network. $\endgroup$ Commented Apr 8 at 6:00

2 Answers 2

19
$\begingroup$

Yes, this is a stereotypical reinforcement learning problem. Instead of trying to calculate the dynamics of the environment the agent is given a reward or punishment for its behavior in the environment (shooting an enemy, etc). The training process tries to find a policy to maximize the reward signal. It’s a rather deep field and a bit much for a single post. Maybe try reading one or two papers on Deep Q Network (DQN) or Proximal policy optimization (PPO) and see if either can be formulated in a way to work for your problem.

$\endgroup$
3
  • 3
    $\begingroup$ If the neural network outputs a sequence of actions, and we only find out at the end whether that entire sequence was successful, it is a RL problem. The problem as stated does not appear to have that form. Instead, the neural network appears to make a single choice (the angle to shoot at) and then we learn whether that was a good choice. That is a simple supervised learning problem -- no need for the machinery of reinforcement learning. $\endgroup$
    – D.W.
    Commented Apr 9 at 1:41
  • 2
    $\begingroup$ It would probably be helpful to spell out the full names of DQN and PPO (Deep Q-Network and Proximal Policy Optimization?). $\endgroup$
    – jcaron
    Commented Apr 9 at 10:50
  • 1
    $\begingroup$ @D.W. I can see your argument there. I took there being a bot and a human to mean that it might be competitive and round based. A supervised learner trained to make a locally optimal choice may miss a broader strategy. $\endgroup$
    – foreverska
    Commented Apr 9 at 12:19
11
$\begingroup$

Yes, but we're talking about a lot of ratings, like, millions of ratings. You have to automate rating generation, human feedback would just get way too expensive.

$\endgroup$
2
  • 2
    $\begingroup$ One alternative would be to have the AI play versus another AI in a reinforcement learning scenario, which is a fairly common strategy. (You can do this in parallel for many hours.) Of course, there are some drawbacks. It gets more complicated to code and they don't always behave like you want them to. If your simulated AIs start taking advantage of edge glitches or decide it's far better to not compete at all, it wouldn't be very useful for normal gameplay. $\endgroup$
    – NotANumber
    Commented Apr 9 at 6:11
  • 1
    $\begingroup$ @NotANumber Sure. You've suggested a very reasonable means of automating it. :) $\endgroup$ Commented Apr 9 at 17:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .