4
$\begingroup$

I'm trying to find the name for a model that is used to output a decision (maybe something like right, left, or do nothing = -1, 0,1) but that can be trained with labels that contain how "correct" or "incorrect" it was. I've tried to google around and ask some friends in my machine learning class, but no one seems to have an answer.

The classic example I seem to always see is the models used in the snake game. We don't know what the right decision was per se, but we can say that if it ran into the wall, that was really wrong. Or if it got an apple and gained 50 points, then it was correct and if it got 2 apples and gained 100 points then it was even more correct, etc.

I'm looking for a network where the exact labels don't exist, but where we can penalize or reward its decisions.

I'm assuming this requires some kind of modified cost function, but I would imagine this type of network already exists. I'm hoping someone can provide me with the name for this type of network and whether or not there is a Keras frontend for something like this.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
13
$\begingroup$

What you are looking for is called "reinforcement learning".

A reinforcement learning algorithm will try to maximize a reward function. This reward represents how "good" or "bad" an action is in the actual context. For example, in the snake game, your reward will be positive for eating an apple and negative when the snake hits a wall.

The interesting thing is that, with reinforcement learning, you can learn without having a reward at each step. In the case of the snake game, your agent can learn that going in the direction of the apple is better than going in the direction of the wall, even if none of this action will directly give a reward (positive or negative).

If you want to use a neural network as your post seem to imply then you should look at deep Q-learning, a reinforcement learning algorithm, which use a neural network to learn to predict the expected reward of a couple (state, action).

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ I agree and also think the OP is looking for RL. It is worth noting in the answer that the neural network can only be part of the RL algorithm. You cannot write a neural network that "does RL" using e.g. Keras. The OP will want to learn the general approach and structure for RL first. A good starting method might be Q-learning and DQN for how neural networks fit it. $\endgroup$
    – Neil Slater
    Feb 11, 2021 at 13:56
  • $\begingroup$ Perfect! Thank you. The terms I was googling were just not giving me "reinforcement learning" haha. I've heard the name many times but just didn't realize that is what it was called. $\endgroup$
    – Kyle Dixon
    Feb 11, 2021 at 17:14
  • 1
    $\begingroup$ @NeilSlater I'm starting to write a DQN following the Keras examples for an actor and critic now. So far it's pretty unique compared to what I'm used to but this seems like a great topic to learn next. $\endgroup$
    – Kyle Dixon
    Feb 11, 2021 at 17:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .