2
$\begingroup$

Let's say our task is to pick and place a block, like: https://gym.openai.com/envs/FetchPickAndPlace-v0/

Reward function 1: -1 for block not placed, 0 for block placed

Reward function 2: 0 for block not placed, +1 for block placed

I noticed training 1 is much faster than 2... I am using the HER implementation from OpenAI. Why is that?

Improve this question
$\endgroup$
3
  • $\begingroup$ Could you please clarify when the rewards are assigned? It makes a big difference to most RL algorithms whether the "block not placed" reward is given on every time step or just at the end as an assssment. $\endgroup$ – Neil Slater Aug 5 '19 at 8:07
  • $\begingroup$ And also, it seems that 1st option has a priori denser rewards. I'm not familiar with the HER implementation, apart from the experience replay, does anyone know if it centers the expected reward somehow? (State functions or GAE) $\endgroup$ – David Aug 5 '19 at 12:07
  • $\begingroup$ These are given at each time step. The 'density' aka frequency of each type of reward is the same for each reward function. $\endgroup$ – user3180 Aug 7 '19 at 4:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.