5
$\begingroup$

I'm confused with the two terminology - action and policy - in Reinforcement Learning. As far as I know, the action is:

It is what the agent makes in a given state.

However, the book I'm reading now (Hands-On Reinforcement Learning with Python) writes the following to explain policy:

we defined the entity that tells us what to do in every state as policy.

Now, I feel that the policy is the same as the action. So what is the difference between the two, and how can I use them apart correctly?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
8
$\begingroup$

A policy is a function that maps states to a probability distribution over all possible actions.

So, in a typical Atari game, there might just be a handful of actions, represented by the keys that are used to play the game. In this context, the policy of a reinforcement learner might be represented by a pretty complex neural network that gets pixels as input and gives action probabilities as output.

Improve this answer
$\endgroup$
2
  • $\begingroup$ @DennisSoemers Why does the epsilon-greedy assign high probability to actions that maximize the predicted returns? Isn't it just what makes an action randomly with the probability of epsilon, and choose actions with large rewards with (1 - epsilon)...? $\endgroup$
    – Blaszard
    Commented Aug 1, 2018 at 19:57
  • 1
    $\begingroup$ @Blaszard Correct. Suppose epsilon = 0.1, and you have 2 actions. There is an 0.9 probability of greedily selecting the best action (or choosing randomly among multiple "equally best" actions), and an 0.1 probability of selecting uniformly at random. In that second case, there is again an 0.5 probability of selecting the best action, and 0.5 probability of selecting the worst (for 2 actions). Mathematically, that can be summarized as a total probability of 0.95 for best action, and 0.05 for worst action. $\endgroup$
    – Dennis Soemers ♦
    Commented Aug 1, 2018 at 20:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .