2
$\begingroup$

Let's say our task is to pick and place a block, like: https://gym.openai.com/envs/FetchPickAndPlace-v0/

Reward function 1: -1 for block not placed, 0 for block placed

Reward function 2: 0 for block not placed, +1 for block placed

I noticed training 1 is much faster than 2... I am using the HER implementation from OpenAI. Why is that?

Improve this question
$\endgroup$
3
  • $\begingroup$ Could you please clarify when the rewards are assigned? It makes a big difference to most RL algorithms whether the "block not placed" reward is given on every time step or just at the end as an assssment. $\endgroup$
    – Neil Slater
    Aug 5, 2019 at 8:07
  • $\begingroup$ And also, it seems that 1st option has a priori denser rewards. I'm not familiar with the HER implementation, apart from the experience replay, does anyone know if it centers the expected reward somehow? (State functions or GAE) $\endgroup$
    – David
    Aug 5, 2019 at 12:07
  • $\begingroup$ These are given at each time step. The 'density' aka frequency of each type of reward is the same for each reward function. $\endgroup$
    – user3180
    Aug 7, 2019 at 4:31

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .