Skip to main content
3 votes
Accepted

What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?

This answer assumes that you only have a problem with this notation from the article: $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ This is a standard notation, used in many ...
Neil Slater's user avatar
  • 32.7k
2 votes

Why does HER not work with on-policy RL algorithms?

In principle replay mechanisms as HER, PER, etc cannot be applied to on-policy algorithms, like SARSA and Policy gradient as stated by @Neil Slater. Anyway, If you are willing to "adapt" ...
Luca Anzalone's user avatar
2 votes
Accepted

Why does HER not work with on-policy RL algorithms?

On policy algorithms contain policy and/or value update calculations that assume data was generated by the current policy. Breaking that assumption will cause them to miscalculate, or not function at ...
Neil Slater's user avatar
  • 32.7k
1 vote
Accepted

A question about the reward calculation in the Hindsight Experience Replay algorithm

Meanwhile I can answer my own question. After figuring it out, what the HER algorithm really needs, I found out, that the right answer is 1: you need a reward function for calculating the new reward ...
Dave's user avatar
  • 172
1 vote
Accepted

What do the state features of KukaGymEnv represent?

Here's an incomplete answer, but it may help. Your state is read by the function getExtendedObservation(). This function makes two things : it calls the function <...
16Aghnar's user avatar
  • 601
1 vote
Accepted

What is the difference between success rate and reward when dealing with binary and sparse rewards?

Page 6 of the paper describes the exact reward functions, and why they were used: Goals: Goals describe the desired position of the object (a box or a puck depending on the task) with some fixed ...
John Doucette's user avatar
1 vote

How does Hindsight Experience Replay learn from unsuccessful trajectories?

Ignoring HER for now the $Q$ and $V$ functions operate on states and actions which are part of a Markov decision process which we call $M_0$. Back to HER, the $Q$ and $V$ functions now take a goal as ...
Jeremy List's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible